{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "359ac63b",
   "metadata": {
    "origin_pos": 0
   },
   "source": [
    "# Multivariable Calculus\n",
    ":label:`sec_multivariable_calculus`\n",
    "\n",
    "Now that we have a fairly strong understanding of derivatives of a function of a single variable, let's return to our original question where we were considering a loss function of potentially billions of weights.\n",
    "\n",
    "## Higher-Dimensional Differentiation\n",
    "What :numref:`sec_single_variable_calculus` tells us is that if we change a single one of these billions of weights leaving every other one fixed, we know what will happen!  This is nothing more than a function of a single variable, so we can write\n",
    "\n",
    "$$L(w_1+\\epsilon_1, w_2, \\ldots, w_N) \\approx L(w_1, w_2, \\ldots, w_N) + \\epsilon_1 \\frac{d}{dw_1} L(w_1, w_2, \\ldots, w_N).$$\n",
    ":eqlabel:`eq_part_der`\n",
    "\n",
    "We will call the derivative in one variable while fixing the other variables the *partial derivative*, and we will use the notation $\\frac{\\partial}{\\partial w_1}$ for the derivative in :eqref:`eq_part_der`.\n",
    "\n",
    "Now, let's take this and change $w_2$ a little bit to $w_2 + \\epsilon_2$:\n",
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    "L(w_1+\\epsilon_1, w_2+\\epsilon_2, \\ldots, w_N) & \\approx L(w_1, w_2+\\epsilon_2, \\ldots, w_N) + \\epsilon_1 \\frac{\\partial}{\\partial w_1} L(w_1, w_2+\\epsilon_2, \\ldots, w_N+\\epsilon_N) \\\\\n",
    "& \\approx L(w_1, w_2, \\ldots, w_N) \\\\\n",
    "& \\quad + \\epsilon_2\\frac{\\partial}{\\partial w_2} L(w_1, w_2, \\ldots, w_N) \\\\\n",
    "& \\quad + \\epsilon_1 \\frac{\\partial}{\\partial w_1} L(w_1, w_2, \\ldots, w_N) \\\\\n",
    "& \\quad + \\epsilon_1\\epsilon_2\\frac{\\partial}{\\partial w_2}\\frac{\\partial}{\\partial w_1} L(w_1, w_2, \\ldots, w_N) \\\\\n",
    "& \\approx L(w_1, w_2, \\ldots, w_N) \\\\\n",
    "& \\quad + \\epsilon_2\\frac{\\partial}{\\partial w_2} L(w_1, w_2, \\ldots, w_N) \\\\\n",
    "& \\quad + \\epsilon_1 \\frac{\\partial}{\\partial w_1} L(w_1, w_2, \\ldots, w_N).\n",
    "\\end{aligned}\n",
    "$$\n",
    "\n",
    "We have again used the idea that $\\epsilon_1\\epsilon_2$ is a higher order term that we can discard in the same way we could discard $\\epsilon^{2}$ in the previous section, along with what we saw in :eqref:`eq_part_der`.  By continuing in this manner, we may write that\n",
    "\n",
    "$$\n",
    "L(w_1+\\epsilon_1, w_2+\\epsilon_2, \\ldots, w_N+\\epsilon_N) \\approx L(w_1, w_2, \\ldots, w_N) + \\sum_i \\epsilon_i \\frac{\\partial}{\\partial w_i} L(w_1, w_2, \\ldots, w_N).\n",
    "$$\n",
    "\n",
    "This may look like a mess, but we can make this more familiar by noting that the sum on the right looks exactly like a dot product, so if we let\n",
    "\n",
    "$$\n",
    "\\boldsymbol{\\epsilon} = [\\epsilon_1, \\ldots, \\epsilon_N]^\\top \\; \\textrm{and} \\;\n",
    "\\nabla_{\\mathbf{x}} L = \\left[\\frac{\\partial L}{\\partial x_1}, \\ldots, \\frac{\\partial L}{\\partial x_N}\\right]^\\top,\n",
    "$$\n",
    "\n",
    "then\n",
    "\n",
    "$$L(\\mathbf{w} + \\boldsymbol{\\epsilon}) \\approx L(\\mathbf{w}) + \\boldsymbol{\\epsilon}\\cdot \\nabla_{\\mathbf{w}} L(\\mathbf{w}).$$\n",
    ":eqlabel:`eq_nabla_use`\n",
    "\n",
    "We will call the vector $\\nabla_{\\mathbf{w}} L$ the *gradient* of $L$.\n",
    "\n",
    "Equation :eqref:`eq_nabla_use` is worth pondering for a moment.  It has exactly the format that we encountered in one dimension, just we have converted everything to vectors and dot products.  It allows us to tell approximately how the function $L$ will change given any perturbation to the input.  As we will see in the next section, this will provide us with an important tool in understanding geometrically how we can learn using information contained in the gradient.\n",
    "\n",
    "But first, let's see this approximation at work with an example.  Suppose that we are working with the function\n",
    "\n",
    "$$\n",
    "f(x, y) = \\log(e^x + e^y) \\textrm{ with gradient } \\nabla f (x, y) = \\left[\\frac{e^x}{e^x+e^y}, \\frac{e^y}{e^x+e^y}\\right].\n",
    "$$\n",
    "\n",
    "If we look at a point like $(0, \\log(2))$, we see that\n",
    "\n",
    "$$\n",
    "f(x, y) = \\log(3) \\textrm{ with gradient } \\nabla f (x, y) = \\left[\\frac{1}{3}, \\frac{2}{3}\\right].\n",
    "$$\n",
    "\n",
    "Thus, if we want to approximate $f$ at $(\\epsilon_1, \\log(2) + \\epsilon_2)$,  we see that we should have the specific instance of :eqref:`eq_nabla_use`:\n",
    "\n",
    "$$\n",
    "f(\\epsilon_1, \\log(2) + \\epsilon_2) \\approx \\log(3) + \\frac{1}{3}\\epsilon_1 + \\frac{2}{3}\\epsilon_2.\n",
    "$$\n",
    "\n",
    "We can test this in code to see how good the approximation is.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "49275f00",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-08-18T19:29:37.326255Z",
     "iopub.status.busy": "2023-08-18T19:29:37.325624Z",
     "iopub.status.idle": "2023-08-18T19:29:40.144730Z",
     "shell.execute_reply": "2023-08-18T19:29:40.143129Z"
    },
    "origin_pos": 2,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'approximation: tensor([1.0819]), true Value: tensor([1.0821])'"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import numpy as np\n",
    "import torch\n",
    "from IPython import display\n",
    "from mpl_toolkits import mplot3d\n",
    "from d2l import torch as d2l\n",
    "\n",
    "\n",
    "def f(x, y):\n",
    "    return torch.log(torch.exp(x) + torch.exp(y))\n",
    "def grad_f(x, y):\n",
    "    return torch.tensor([torch.exp(x) / (torch.exp(x) + torch.exp(y)),\n",
    "                     torch.exp(y) / (torch.exp(x) + torch.exp(y))])\n",
    "\n",
    "epsilon = torch.tensor([0.01, -0.03])\n",
    "grad_approx = f(torch.tensor([0.]), torch.log(\n",
    "    torch.tensor([2.]))) + epsilon.dot(\n",
    "    grad_f(torch.tensor([0.]), torch.log(torch.tensor(2.))))\n",
    "true_value = f(torch.tensor([0.]) + epsilon[0], torch.log(\n",
    "    torch.tensor([2.])) + epsilon[1])\n",
    "f'approximation: {grad_approx}, true Value: {true_value}'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2899e6c8",
   "metadata": {
    "origin_pos": 4
   },
   "source": [
    "## Geometry of Gradients and Gradient Descent\n",
    "Consider the expression from :eqref:`eq_nabla_use` again:\n",
    "\n",
    "$$\n",
    "L(\\mathbf{w} + \\boldsymbol{\\epsilon}) \\approx L(\\mathbf{w}) + \\boldsymbol{\\epsilon}\\cdot \\nabla_{\\mathbf{w}} L(\\mathbf{w}).\n",
    "$$\n",
    "\n",
    "Let's suppose that I want to use this to help minimize our loss $L$.  Let's understand geometrically the algorithm of gradient descent first described in  :numref:`sec_autograd`. What we will do is the following:\n",
    "\n",
    "1. Start with a random choice for the initial parameters $\\mathbf{w}$.\n",
    "2. Find the direction $\\mathbf{v}$ that makes $L$ decrease the most rapidly at $\\mathbf{w}$.\n",
    "3. Take a small step in that direction: $\\mathbf{w} \\rightarrow \\mathbf{w} + \\epsilon\\mathbf{v}$.\n",
    "4. Repeat.\n",
    "\n",
    "The only thing we do not know exactly how to do is to compute the vector $\\mathbf{v}$ in the second step.  We will call such a direction the *direction of steepest descent*.  Using the geometric understanding of dot products from :numref:`sec_geometry-linear-algebraic-ops`, we see that we can rewrite :eqref:`eq_nabla_use` as\n",
    "\n",
    "$$\n",
    "L(\\mathbf{w} + \\mathbf{v}) \\approx L(\\mathbf{w}) + \\mathbf{v}\\cdot \\nabla_{\\mathbf{w}} L(\\mathbf{w}) = L(\\mathbf{w}) + \\|\\nabla_{\\mathbf{w}} L(\\mathbf{w})\\|\\cos(\\theta).\n",
    "$$\n",
    "\n",
    "Note that we have taken our direction to have length one for convenience, and used $\\theta$ for the angle between $\\mathbf{v}$ and $\\nabla_{\\mathbf{w}} L(\\mathbf{w})$.  If we want to find the direction that decreases $L$ as rapidly as possible, we want to make this expression as negative as possible.  The only way the direction we pick enters into this equation is through $\\cos(\\theta)$, and thus we wish to make this cosine as negative as possible.  Now, recalling the shape of cosine, we can make this as negative as possible by making $\\cos(\\theta) = -1$ or equivalently making the angle between the gradient and our chosen direction to be $\\pi$ radians, or equivalently $180$ degrees.  The only way to achieve this is to head in the exact opposite direction:  pick $\\mathbf{v}$ to point in the exact opposite direction to $\\nabla_{\\mathbf{w}} L(\\mathbf{w})$!\n",
    "\n",
    "This brings us to one of the most important mathematical concepts in machine learning: the direction of steepest decent points in the direction of $-\\nabla_{\\mathbf{w}}L(\\mathbf{w})$.  Thus our informal algorithm can be rewritten as follows.\n",
    "\n",
    "1. Start with a random choice for the initial parameters $\\mathbf{w}$.\n",
    "2. Compute $\\nabla_{\\mathbf{w}} L(\\mathbf{w})$.\n",
    "3. Take a small step in the opposite of that direction: $\\mathbf{w} \\leftarrow \\mathbf{w} - \\epsilon\\nabla_{\\mathbf{w}} L(\\mathbf{w})$.\n",
    "4. Repeat.\n",
    "\n",
    "\n",
    "This basic algorithm has been modified and adapted many ways by many researchers, but the core concept remains the same in all of them.  Use the gradient to find the direction that decreases the loss as rapidly as possible, and update the parameters to take a step in that direction.\n",
    "\n",
    "## A Note on Mathematical Optimization\n",
    "Throughout this book, we focus squarely on numerical optimization techniques for the practical reason that all functions we encounter in the deep learning setting are too complex to minimize explicitly.\n",
    "\n",
    "However, it is a useful exercise to consider what the geometric understanding we obtained above tells us about optimizing functions directly.\n",
    "\n",
    "Suppose that we wish to find the value of $\\mathbf{x}_0$ which minimizes some function $L(\\mathbf{x})$.  Let's suppose that moreover someone gives us a value and tells us that it is the value that minimizes $L$.  Is there anything we can check to see if their answer is even plausible?\n",
    "\n",
    "Again consider :eqref:`eq_nabla_use`:\n",
    "$$\n",
    "L(\\mathbf{x}_0 + \\boldsymbol{\\epsilon}) \\approx L(\\mathbf{x}_0) + \\boldsymbol{\\epsilon}\\cdot \\nabla_{\\mathbf{x}} L(\\mathbf{x}_0).\n",
    "$$\n",
    "\n",
    "If the gradient is not zero, we know that we can take a step in the direction $-\\epsilon \\nabla_{\\mathbf{x}} L(\\mathbf{x}_0)$ to find a value of $L$ that is smaller.  Thus, if we truly are at a minimum, this cannot be the case!  We can conclude that if $\\mathbf{x}_0$ is a minimum, then $\\nabla_{\\mathbf{x}} L(\\mathbf{x}_0) = 0$.  We call points with $\\nabla_{\\mathbf{x}} L(\\mathbf{x}_0) = 0$ *critical points*.\n",
    "\n",
    "This is nice, because in some rare settings, we *can* explicitly find all the points where the gradient is zero, and find the one with the smallest value.\n",
    "\n",
    "For a concrete example, consider the function\n",
    "$$\n",
    "f(x) = 3x^4 - 4x^3 -12x^2.\n",
    "$$\n",
    "\n",
    "This function has derivative\n",
    "$$\n",
    "\\frac{df}{dx} = 12x^3 - 12x^2 -24x = 12x(x-2)(x+1).\n",
    "$$\n",
    "\n",
    "The only possible location of minima are at $x = -1, 0, 2$, where the function takes the values $-5,0, -32$ respectively, and thus we can conclude that we minimize our function when $x = 2$.  A quick plot confirms this.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "e958293a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-08-18T19:29:40.149841Z",
     "iopub.status.busy": "2023-08-18T19:29:40.148912Z",
     "iopub.status.idle": "2023-08-18T19:29:40.373778Z",
     "shell.execute_reply": "2023-08-18T19:29:40.372858Z"
    },
    "origin_pos": 6,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"251.482813pt\" height=\"183.35625pt\" viewBox=\"0 0 251.482813 183.35625\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
       " <metadata>\n",
       "  <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
       "   <cc:Work>\n",
       "    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
       "    <dc:date>2023-08-18T19:29:40.340259</dc:date>\n",
       "    <dc:format>image/svg+xml</dc:format>\n",
       "    <dc:creator>\n",
       "     <cc:Agent>\n",
       "      <dc:title>Matplotlib v3.7.2, https://matplotlib.org/</dc:title>\n",
       "     </cc:Agent>\n",
       "    </dc:creator>\n",
       "   </cc:Work>\n",
       "  </rdf:RDF>\n",
       " </metadata>\n",
       " <defs>\n",
       "  <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M -0 183.35625 \n",
       "L 251.482813 183.35625 \n",
       "L 251.482813 0 \n",
       "L -0 0 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 48.982813 145.8 \n",
       "L 244.282813 145.8 \n",
       "L 244.282813 7.2 \n",
       "L 48.982813 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_1\">\n",
       "      <path d=\"M 57.860085 145.8 \n",
       "L 57.860085 7.2 \n",
       "\" clip-path=\"url(#p2a4769e885)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_2\">\n",
       "      <defs>\n",
       "       <path id=\"m0ac3209c98\" d=\"M 0 0 \n",
       "L 0 3.5 \n",
       "\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use xlink:href=\"#m0ac3209c98\" x=\"57.860085\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- −2 -->\n",
       "      <g transform=\"translate(50.488991 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-2212\" d=\"M 678 2272 \n",
       "L 4684 2272 \n",
       "L 4684 1741 \n",
       "L 678 1741 \n",
       "L 678 2272 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "        <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
       "L 3431 531 \n",
       "L 3431 0 \n",
       "L 469 0 \n",
       "L 469 531 \n",
       "Q 828 903 1448 1529 \n",
       "Q 2069 2156 2228 2338 \n",
       "Q 2531 2678 2651 2914 \n",
       "Q 2772 3150 2772 3378 \n",
       "Q 2772 3750 2511 3984 \n",
       "Q 2250 4219 1831 4219 \n",
       "Q 1534 4219 1204 4116 \n",
       "Q 875 4013 500 3803 \n",
       "L 500 4441 \n",
       "Q 881 4594 1212 4672 \n",
       "Q 1544 4750 1819 4750 \n",
       "Q 2544 4750 2975 4387 \n",
       "Q 3406 4025 3406 3419 \n",
       "Q 3406 3131 3298 2873 \n",
       "Q 3191 2616 2906 2266 \n",
       "Q 2828 2175 2409 1742 \n",
       "Q 1991 1309 1228 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-2212\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-32\" x=\"83.789062\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_3\">\n",
       "      <path d=\"M 93.440337 145.8 \n",
       "L 93.440337 7.2 \n",
       "\" clip-path=\"url(#p2a4769e885)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_4\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m0ac3209c98\" x=\"93.440337\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- −1 -->\n",
       "      <g transform=\"translate(86.069243 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
       "L 1825 531 \n",
       "L 1825 4091 \n",
       "L 703 3866 \n",
       "L 703 4441 \n",
       "L 1819 4666 \n",
       "L 2450 4666 \n",
       "L 2450 531 \n",
       "L 3481 531 \n",
       "L 3481 0 \n",
       "L 794 0 \n",
       "L 794 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-2212\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-31\" x=\"83.789062\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_5\">\n",
       "      <path d=\"M 129.020588 145.8 \n",
       "L 129.020588 7.2 \n",
       "\" clip-path=\"url(#p2a4769e885)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m0ac3209c98\" x=\"129.020588\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- 0 -->\n",
       "      <g transform=\"translate(125.839338 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
       "Q 1547 4250 1301 3770 \n",
       "Q 1056 3291 1056 2328 \n",
       "Q 1056 1369 1301 889 \n",
       "Q 1547 409 2034 409 \n",
       "Q 2525 409 2770 889 \n",
       "Q 3016 1369 3016 2328 \n",
       "Q 3016 3291 2770 3770 \n",
       "Q 2525 4250 2034 4250 \n",
       "z\n",
       "M 2034 4750 \n",
       "Q 2819 4750 3233 4129 \n",
       "Q 3647 3509 3647 2328 \n",
       "Q 3647 1150 3233 529 \n",
       "Q 2819 -91 2034 -91 \n",
       "Q 1250 -91 836 529 \n",
       "Q 422 1150 422 2328 \n",
       "Q 422 3509 836 4129 \n",
       "Q 1250 4750 2034 4750 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_7\">\n",
       "      <path d=\"M 164.600839 145.8 \n",
       "L 164.600839 7.2 \n",
       "\" clip-path=\"url(#p2a4769e885)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_8\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m0ac3209c98\" x=\"164.600839\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- 1 -->\n",
       "      <g transform=\"translate(161.419589 160.398438) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_5\">\n",
       "     <g id=\"line2d_9\">\n",
       "      <path d=\"M 200.181091 145.8 \n",
       "L 200.181091 7.2 \n",
       "\" clip-path=\"url(#p2a4769e885)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_10\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m0ac3209c98\" x=\"200.181091\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_5\">\n",
       "      <!-- 2 -->\n",
       "      <g transform=\"translate(196.999841 160.398438) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_6\">\n",
       "     <g id=\"line2d_11\">\n",
       "      <path d=\"M 235.761342 145.8 \n",
       "L 235.761342 7.2 \n",
       "\" clip-path=\"url(#p2a4769e885)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_12\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m0ac3209c98\" x=\"235.761342\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_6\">\n",
       "      <!-- 3 -->\n",
       "      <g transform=\"translate(232.580092 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-33\" d=\"M 2597 2516 \n",
       "Q 3050 2419 3304 2112 \n",
       "Q 3559 1806 3559 1356 \n",
       "Q 3559 666 3084 287 \n",
       "Q 2609 -91 1734 -91 \n",
       "Q 1441 -91 1130 -33 \n",
       "Q 819 25 488 141 \n",
       "L 488 750 \n",
       "Q 750 597 1062 519 \n",
       "Q 1375 441 1716 441 \n",
       "Q 2309 441 2620 675 \n",
       "Q 2931 909 2931 1356 \n",
       "Q 2931 1769 2642 2001 \n",
       "Q 2353 2234 1838 2234 \n",
       "L 1294 2234 \n",
       "L 1294 2753 \n",
       "L 1863 2753 \n",
       "Q 2328 2753 2575 2939 \n",
       "Q 2822 3125 2822 3475 \n",
       "Q 2822 3834 2567 4026 \n",
       "Q 2313 4219 1838 4219 \n",
       "Q 1578 4219 1281 4162 \n",
       "Q 984 4106 628 3988 \n",
       "L 628 4550 \n",
       "Q 988 4650 1302 4700 \n",
       "Q 1616 4750 1894 4750 \n",
       "Q 2613 4750 3031 4423 \n",
       "Q 3450 4097 3450 3541 \n",
       "Q 3450 3153 3228 2886 \n",
       "Q 3006 2619 2597 2516 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-33\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_7\">\n",
       "     <!-- x -->\n",
       "     <g transform=\"translate(143.673438 174.076563) scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-78\" d=\"M 3513 3500 \n",
       "L 2247 1797 \n",
       "L 3578 0 \n",
       "L 2900 0 \n",
       "L 1881 1375 \n",
       "L 863 0 \n",
       "L 184 0 \n",
       "L 1544 1831 \n",
       "L 300 3500 \n",
       "L 978 3500 \n",
       "L 1906 2253 \n",
       "L 2834 3500 \n",
       "L 3513 3500 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-78\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_13\">\n",
       "      <path d=\"M 48.982813 115.875 \n",
       "L 244.282813 115.875 \n",
       "\" clip-path=\"url(#p2a4769e885)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_14\">\n",
       "      <defs>\n",
       "       <path id=\"m27cfc7b42c\" d=\"M 0 0 \n",
       "L -3.5 0 \n",
       "\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use xlink:href=\"#m27cfc7b42c\" x=\"48.982813\" y=\"115.875\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_8\">\n",
       "      <!-- −20 -->\n",
       "      <g transform=\"translate(20.878125 119.674219) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-2212\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-32\" x=\"83.789062\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"147.412109\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_15\">\n",
       "      <path d=\"M 48.982813 76.5 \n",
       "L 244.282813 76.5 \n",
       "\" clip-path=\"url(#p2a4769e885)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_16\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m27cfc7b42c\" x=\"48.982813\" y=\"76.5\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- 0 -->\n",
       "      <g transform=\"translate(35.620313 80.299219) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_17\">\n",
       "      <path d=\"M 48.982813 37.125 \n",
       "L 244.282813 37.125 \n",
       "\" clip-path=\"url(#p2a4769e885)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_18\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m27cfc7b42c\" x=\"48.982813\" y=\"37.125\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_10\">\n",
       "      <!-- 20 -->\n",
       "      <g transform=\"translate(29.257813 40.924219) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_11\">\n",
       "     <!-- f(x) -->\n",
       "     <g transform=\"translate(14.798438 85.121094) rotate(-90) scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-66\" d=\"M 2375 4863 \n",
       "L 2375 4384 \n",
       "L 1825 4384 \n",
       "Q 1516 4384 1395 4259 \n",
       "Q 1275 4134 1275 3809 \n",
       "L 1275 3500 \n",
       "L 2222 3500 \n",
       "L 2222 3053 \n",
       "L 1275 3053 \n",
       "L 1275 0 \n",
       "L 697 0 \n",
       "L 697 3053 \n",
       "L 147 3053 \n",
       "L 147 3500 \n",
       "L 697 3500 \n",
       "L 697 3744 \n",
       "Q 697 4328 969 4595 \n",
       "Q 1241 4863 1831 4863 \n",
       "L 2375 4863 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-28\" d=\"M 1984 4856 \n",
       "Q 1566 4138 1362 3434 \n",
       "Q 1159 2731 1159 2009 \n",
       "Q 1159 1288 1364 580 \n",
       "Q 1569 -128 1984 -844 \n",
       "L 1484 -844 \n",
       "Q 1016 -109 783 600 \n",
       "Q 550 1309 550 2009 \n",
       "Q 550 2706 781 3412 \n",
       "Q 1013 4119 1484 4856 \n",
       "L 1984 4856 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-29\" d=\"M 513 4856 \n",
       "L 1013 4856 \n",
       "Q 1481 4119 1714 3412 \n",
       "Q 1947 2706 1947 2009 \n",
       "Q 1947 1309 1714 600 \n",
       "Q 1481 -109 1013 -844 \n",
       "L 513 -844 \n",
       "Q 928 -128 1133 580 \n",
       "Q 1338 1288 1338 2009 \n",
       "Q 1338 2731 1133 3434 \n",
       "Q 928 4138 513 4856 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-66\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-28\" x=\"35.205078\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-78\" x=\"74.21875\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-29\" x=\"133.398438\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"line2d_19\">\n",
       "    <path d=\"M 57.860085 13.5 \n",
       "L 59.994898 24.256478 \n",
       "L 62.129716 33.892624 \n",
       "L 64.264529 42.477134 \n",
       "L 66.399346 50.076855 \n",
       "L 68.534159 56.756774 \n",
       "L 70.313174 61.666423 \n",
       "L 72.092185 66.016803 \n",
       "L 73.871196 69.842626 \n",
       "L 75.650211 73.177734 \n",
       "L 77.429226 76.055025 \n",
       "L 79.208233 78.506549 \n",
       "L 80.987248 80.563461 \n",
       "L 82.766259 82.256031 \n",
       "L 84.545274 83.613647 \n",
       "L 86.324285 84.664799 \n",
       "L 88.103295 85.437102 \n",
       "L 89.882311 85.957283 \n",
       "L 91.661326 86.251179 \n",
       "L 93.440339 86.343749 \n",
       "L 95.575152 86.222902 \n",
       "L 97.709967 85.886658 \n",
       "L 100.200584 85.272816 \n",
       "L 103.047004 84.348982 \n",
       "L 106.960832 82.83189 \n",
       "L 116.923302 78.842604 \n",
       "L 120.125525 77.830444 \n",
       "L 122.971945 77.13914 \n",
       "L 125.818365 76.685234 \n",
       "L 128.308983 76.509386 \n",
       "L 130.799601 76.56001 \n",
       "L 133.290218 76.852583 \n",
       "L 135.780836 77.39918 \n",
       "L 138.271453 78.208471 \n",
       "L 140.76207 79.285723 \n",
       "L 143.252689 80.6328 \n",
       "L 145.743306 82.248162 \n",
       "L 148.233922 84.126866 \n",
       "L 151.080344 86.585555 \n",
       "L 153.926763 89.359284 \n",
       "L 157.128987 92.826557 \n",
       "L 160.331209 96.619841 \n",
       "L 164.245035 101.622445 \n",
       "L 169.226272 108.399591 \n",
       "L 180.611954 124.070876 \n",
       "L 184.169976 128.493666 \n",
       "L 187.016397 131.680977 \n",
       "L 189.507013 134.136541 \n",
       "L 191.64183 135.942328 \n",
       "L 193.420841 137.20378 \n",
       "L 195.199856 138.216626 \n",
       "L 196.978871 138.954239 \n",
       "L 198.402075 139.327694 \n",
       "L 199.825288 139.492948 \n",
       "L 201.248497 139.435142 \n",
       "L 202.671706 139.139076 \n",
       "L 204.094915 138.589142 \n",
       "L 205.518123 137.769443 \n",
       "L 206.94134 136.663643 \n",
       "L 208.364549 135.255108 \n",
       "L 209.787758 133.526797 \n",
       "L 211.210966 131.461346 \n",
       "L 212.634175 129.040993 \n",
       "L 214.057384 126.247625 \n",
       "L 215.836403 122.203127 \n",
       "L 217.615414 117.509975 \n",
       "L 219.394425 112.130497 \n",
       "L 221.173436 106.025933 \n",
       "L 222.952455 99.156757 \n",
       "L 224.731466 91.482579 \n",
       "L 226.866279 81.15271 \n",
       "L 229.001092 69.530999 \n",
       "L 231.135905 56.542494 \n",
       "L 233.270727 42.110486 \n",
       "L 235.40554 26.156299 \n",
       "L 235.40554 26.156299 \n",
       "\" clip-path=\"url(#p2a4769e885)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 48.982813 145.8 \n",
       "L 48.982813 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 244.282813 145.8 \n",
       "L 244.282813 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 48.982813 145.8 \n",
       "L 244.282813 145.8 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 48.982813 7.2 \n",
       "L 244.282813 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"p2a4769e885\">\n",
       "   <rect x=\"48.982813\" y=\"7.2\" width=\"195.3\" height=\"138.6\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<Figure size 350x250 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = torch.arange(-2, 3, 0.01)\n",
    "f = (3 * x**4) - (4 * x**3) - (12 * x**2)\n",
    "\n",
    "d2l.plot(x, f, 'x', 'f(x)')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a6bb7da9",
   "metadata": {
    "origin_pos": 8
   },
   "source": [
    "This highlights an important fact to know when working either theoretically or numerically: the only possible points where we can minimize (or maximize) a function will have gradient equal to zero, however, not every point with gradient zero is the true *global* minimum (or maximum).\n",
    "\n",
    "## Multivariate Chain Rule\n",
    "Let's suppose that we have a function of four variables ($w, x, y$, and $z$) which we can make by composing many terms:\n",
    "\n",
    "$$\\begin{aligned}f(u, v) & = (u+v)^{2} \\\\u(a, b) & = (a+b)^{2}, \\qquad v(a, b) = (a-b)^{2}, \\\\a(w, x, y, z) & = (w+x+y+z)^{2}, \\qquad b(w, x, y, z) = (w+x-y-z)^2.\\end{aligned}$$\n",
    ":eqlabel:`eq_multi_func_def`\n",
    "\n",
    "Such chains of equations are common when working with neural networks, so trying to understand how to compute gradients of such functions is key.  We can start to see visual hints of this connection in :numref:`fig_chain-1` if we take a look at what variables directly relate to one another.\n",
    "\n",
    "![The function relations above where nodes represent values and edges show functional dependence.](../img/chain-net1.svg)\n",
    ":label:`fig_chain-1`\n",
    "\n",
    "Nothing stops us from just composing everything from :eqref:`eq_multi_func_def` and writing out that\n",
    "\n",
    "$$\n",
    "f(w, x, y, z) = \\left(\\left((w+x+y+z)^2+(w+x-y-z)^2\\right)^2+\\left((w+x+y+z)^2-(w+x-y-z)^2\\right)^2\\right)^2.\n",
    "$$\n",
    "\n",
    "We may then take the derivative by just using single variable derivatives, but if we did that we would quickly find ourself swamped with terms, many of which are repeats!  Indeed, one can see that, for instance:\n",
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    "\\frac{\\partial f}{\\partial w} & = 2 \\left(2 \\left(2 (w + x + y + z) - 2 (w + x - y - z)\\right) \\left((w + x + y + z)^{2}- (w + x - y - z)^{2}\\right) + \\right.\\\\\n",
    "& \\left. \\quad 2 \\left(2 (w + x - y - z) + 2 (w + x + y + z)\\right) \\left((w + x - y - z)^{2}+ (w + x + y + z)^{2}\\right)\\right) \\times \\\\\n",
    "& \\quad \\left(\\left((w + x + y + z)^{2}- (w + x - y - z)^2\\right)^{2}+ \\left((w + x - y - z)^{2}+ (w + x + y + z)^{2}\\right)^{2}\\right).\n",
    "\\end{aligned}\n",
    "$$\n",
    "\n",
    "If we then also wanted to compute $\\frac{\\partial f}{\\partial x}$, we would end up with a similar equation again with many repeated terms, and many *shared* repeated terms between the two derivatives.  This represents a massive quantity of wasted work, and if we needed to compute derivatives this way, the whole deep learning revolution would have stalled out before it began!\n",
    "\n",
    "\n",
    "Let's break up the problem.  We will start by trying to understand how $f$ changes when we change $a$, essentially assuming that $w, x, y$, and $z$ all do not exist.  We will reason as we did back when we worked with the gradient for the first time.  Let's take $a$ and add a small amount $\\epsilon$ to it.\n",
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    "& f(u(a+\\epsilon, b), v(a+\\epsilon, b)) \\\\\n",
    "\\approx & f\\left(u(a, b) + \\epsilon\\frac{\\partial u}{\\partial a}(a, b), v(a, b) + \\epsilon\\frac{\\partial v}{\\partial a}(a, b)\\right) \\\\\n",
    "\\approx & f(u(a, b), v(a, b)) + \\epsilon\\left[\\frac{\\partial f}{\\partial u}(u(a, b), v(a, b))\\frac{\\partial u}{\\partial a}(a, b) + \\frac{\\partial f}{\\partial v}(u(a, b), v(a, b))\\frac{\\partial v}{\\partial a}(a, b)\\right].\n",
    "\\end{aligned}\n",
    "$$\n",
    "\n",
    "The first line follows from the definition of partial derivative, and the second follows from the definition of gradient.  It is notationally burdensome to track exactly where we evaluate every derivative, as in the expression $\\frac{\\partial f}{\\partial u}(u(a, b), v(a, b))$, so we often abbreviate this to the much more memorable\n",
    "\n",
    "$$\n",
    "\\frac{\\partial f}{\\partial a} = \\frac{\\partial f}{\\partial u}\\frac{\\partial u}{\\partial a}+\\frac{\\partial f}{\\partial v}\\frac{\\partial v}{\\partial a}.\n",
    "$$\n",
    "\n",
    "It is useful to think about the meaning of the process. We are trying to understand how a function of the form $f(u(a, b), v(a, b))$ changes its value with a change in $a$.  There are two pathways this can occur: there is the pathway where $a \\rightarrow u \\rightarrow f$ and where $a \\rightarrow v \\rightarrow f$.  We can compute both of these contributions via the chain rule: $\\frac{\\partial w}{\\partial u} \\cdot \\frac{\\partial u}{\\partial x}$ and $\\frac{\\partial w}{\\partial v} \\cdot \\frac{\\partial v}{\\partial x}$ respectively, and added up.\n",
    "\n",
    "Imagine we have a different network of functions where the functions on the right depend on those that are connected to on the left as is shown in :numref:`fig_chain-2`.\n",
    "\n",
    "![Another more subtle example of the chain rule.](../img/chain-net2.svg)\n",
    ":label:`fig_chain-2`\n",
    "\n",
    "To compute something like $\\frac{\\partial f}{\\partial y}$, we need to sum over all (in this case $3$) paths from $y$ to $f$ giving\n",
    "\n",
    "$$\n",
    "\\frac{\\partial f}{\\partial y} = \\frac{\\partial f}{\\partial a} \\frac{\\partial a}{\\partial u} \\frac{\\partial u}{\\partial y} + \\frac{\\partial f}{\\partial u} \\frac{\\partial u}{\\partial y} + \\frac{\\partial f}{\\partial b} \\frac{\\partial b}{\\partial v} \\frac{\\partial v}{\\partial y}.\n",
    "$$\n",
    "\n",
    "Understanding the chain rule in this way will pay great dividends when trying to understand how gradients flow through networks, and why various architectural choices like those in LSTMs (:numref:`sec_lstm`) or residual layers (:numref:`sec_resnet`) can help shape the learning process by controlling gradient flow.\n",
    "\n",
    "## The Backpropagation Algorithm\n",
    "\n",
    "Let's return to the example of :eqref:`eq_multi_func_def` the previous section where\n",
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    "f(u, v) & = (u+v)^{2} \\\\\n",
    "u(a, b) & = (a+b)^{2}, \\qquad v(a, b) = (a-b)^{2}, \\\\\n",
    "a(w, x, y, z) & = (w+x+y+z)^{2}, \\qquad b(w, x, y, z) = (w+x-y-z)^2.\n",
    "\\end{aligned}\n",
    "$$\n",
    "\n",
    "If we want to compute say $\\frac{\\partial f}{\\partial w}$ we may apply the multi-variate chain rule to see:\n",
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    "\\frac{\\partial f}{\\partial w} & = \\frac{\\partial f}{\\partial u}\\frac{\\partial u}{\\partial w} + \\frac{\\partial f}{\\partial v}\\frac{\\partial v}{\\partial w}, \\\\\n",
    "\\frac{\\partial u}{\\partial w} & = \\frac{\\partial u}{\\partial a}\\frac{\\partial a}{\\partial w}+\\frac{\\partial u}{\\partial b}\\frac{\\partial b}{\\partial w}, \\\\\n",
    "\\frac{\\partial v}{\\partial w} & = \\frac{\\partial v}{\\partial a}\\frac{\\partial a}{\\partial w}+\\frac{\\partial v}{\\partial b}\\frac{\\partial b}{\\partial w}.\n",
    "\\end{aligned}\n",
    "$$\n",
    "\n",
    "Let's try using this decomposition to compute $\\frac{\\partial f}{\\partial w}$.  Notice that all we need here are the various single step partials:\n",
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    "\\frac{\\partial f}{\\partial u} = 2(u+v), & \\quad\\frac{\\partial f}{\\partial v} = 2(u+v), \\\\\n",
    "\\frac{\\partial u}{\\partial a} = 2(a+b), & \\quad\\frac{\\partial u}{\\partial b} = 2(a+b), \\\\\n",
    "\\frac{\\partial v}{\\partial a} = 2(a-b), & \\quad\\frac{\\partial v}{\\partial b} = -2(a-b), \\\\\n",
    "\\frac{\\partial a}{\\partial w} = 2(w+x+y+z), & \\quad\\frac{\\partial b}{\\partial w} = 2(w+x-y-z).\n",
    "\\end{aligned}\n",
    "$$\n",
    "\n",
    "If we write this out into code this becomes a fairly manageable expression.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "963fe94e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-08-18T19:29:40.377440Z",
     "iopub.status.busy": "2023-08-18T19:29:40.376715Z",
     "iopub.status.idle": "2023-08-18T19:29:40.384350Z",
     "shell.execute_reply": "2023-08-18T19:29:40.383556Z"
    },
    "origin_pos": 9,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    f at -1, 0, -2, 1 is 1024\n",
      "df/dw at -1, 0, -2, 1 is -4096\n"
     ]
    }
   ],
   "source": [
    "# Compute the value of the function from inputs to outputs\n",
    "w, x, y, z = -1, 0, -2, 1\n",
    "a, b = (w + x + y + z)**2, (w + x - y - z)**2\n",
    "u, v = (a + b)**2, (a - b)**2\n",
    "f = (u + v)**2\n",
    "print(f'    f at {w}, {x}, {y}, {z} is {f}')\n",
    "\n",
    "# Compute the single step partials\n",
    "df_du, df_dv = 2*(u + v), 2*(u + v)\n",
    "du_da, du_db, dv_da, dv_db = 2*(a + b), 2*(a + b), 2*(a - b), -2*(a - b)\n",
    "da_dw, db_dw = 2*(w + x + y + z), 2*(w + x - y - z)\n",
    "\n",
    "# Compute the final result from inputs to outputs\n",
    "du_dw, dv_dw = du_da*da_dw + du_db*db_dw, dv_da*da_dw + dv_db*db_dw\n",
    "df_dw = df_du*du_dw + df_dv*dv_dw\n",
    "print(f'df/dw at {w}, {x}, {y}, {z} is {df_dw}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "65532ef3",
   "metadata": {
    "origin_pos": 10
   },
   "source": [
    "However, note that this still does not make it easy to compute something like $\\frac{\\partial f}{\\partial x}$.  The reason for that is the *way* we chose to apply the chain rule.  If we look at what we did above, we always kept $\\partial w$ in the denominator when we could.  In this way, we chose to apply the chain rule seeing how $w$ changed every other variable.  If that is what we wanted, this would be a good idea.  However, think back to our motivation from deep learning: we want to see how every parameter changes the *loss*.  In essence, we want to apply the chain rule keeping $\\partial f$ in the numerator whenever we can!\n",
    "\n",
    "To be more explicit, note that we can write\n",
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    "\\frac{\\partial f}{\\partial w} & = \\frac{\\partial f}{\\partial a}\\frac{\\partial a}{\\partial w} + \\frac{\\partial f}{\\partial b}\\frac{\\partial b}{\\partial w}, \\\\\n",
    "\\frac{\\partial f}{\\partial a} & = \\frac{\\partial f}{\\partial u}\\frac{\\partial u}{\\partial a}+\\frac{\\partial f}{\\partial v}\\frac{\\partial v}{\\partial a}, \\\\\n",
    "\\frac{\\partial f}{\\partial b} & = \\frac{\\partial f}{\\partial u}\\frac{\\partial u}{\\partial b}+\\frac{\\partial f}{\\partial v}\\frac{\\partial v}{\\partial b}.\n",
    "\\end{aligned}\n",
    "$$\n",
    "\n",
    "Note that this application of the chain rule has us explicitly compute $\\frac{\\partial f}{\\partial u}, \\frac{\\partial f}{\\partial v}, \\frac{\\partial f}{\\partial a}, \\frac{\\partial f}{\\partial b}, \\; \\textrm{and} \\; \\frac{\\partial f}{\\partial w}$.  Nothing stops us from also including the equations:\n",
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    "\\frac{\\partial f}{\\partial x} & = \\frac{\\partial f}{\\partial a}\\frac{\\partial a}{\\partial x} + \\frac{\\partial f}{\\partial b}\\frac{\\partial b}{\\partial x}, \\\\\n",
    "\\frac{\\partial f}{\\partial y} & = \\frac{\\partial f}{\\partial a}\\frac{\\partial a}{\\partial y}+\\frac{\\partial f}{\\partial b}\\frac{\\partial b}{\\partial y}, \\\\\n",
    "\\frac{\\partial f}{\\partial z} & = \\frac{\\partial f}{\\partial a}\\frac{\\partial a}{\\partial z}+\\frac{\\partial f}{\\partial b}\\frac{\\partial b}{\\partial z}.\n",
    "\\end{aligned}\n",
    "$$\n",
    "\n",
    "and then keeping track of how $f$ changes when we change *any* node in the entire network.  Let's implement it.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "6236d864",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-08-18T19:29:40.387773Z",
     "iopub.status.busy": "2023-08-18T19:29:40.387181Z",
     "iopub.status.idle": "2023-08-18T19:29:40.396145Z",
     "shell.execute_reply": "2023-08-18T19:29:40.395332Z"
    },
    "origin_pos": 11,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "f at -1, 0, -2, 1 is 1024\n",
      "df/dw at -1, 0, -2, 1 is -4096\n",
      "df/dx at -1, 0, -2, 1 is -4096\n",
      "df/dy at -1, 0, -2, 1 is -4096\n",
      "df/dz at -1, 0, -2, 1 is -4096\n"
     ]
    }
   ],
   "source": [
    "# Compute the value of the function from inputs to outputs\n",
    "w, x, y, z = -1, 0, -2, 1\n",
    "a, b = (w + x + y + z)**2, (w + x - y - z)**2\n",
    "u, v = (a + b)**2, (a - b)**2\n",
    "f = (u + v)**2\n",
    "print(f'f at {w}, {x}, {y}, {z} is {f}')\n",
    "\n",
    "# Compute the derivative using the decomposition above\n",
    "# First compute the single step partials\n",
    "df_du, df_dv = 2*(u + v), 2*(u + v)\n",
    "du_da, du_db, dv_da, dv_db = 2*(a + b), 2*(a + b), 2*(a - b), -2*(a - b)\n",
    "da_dw, db_dw = 2*(w + x + y + z), 2*(w + x - y - z)\n",
    "da_dx, db_dx = 2*(w + x + y + z), 2*(w + x - y - z)\n",
    "da_dy, db_dy = 2*(w + x + y + z), -2*(w + x - y - z)\n",
    "da_dz, db_dz = 2*(w + x + y + z), -2*(w + x - y - z)\n",
    "\n",
    "# Now compute how f changes when we change any value from output to input\n",
    "df_da, df_db = df_du*du_da + df_dv*dv_da, df_du*du_db + df_dv*dv_db\n",
    "df_dw, df_dx = df_da*da_dw + df_db*db_dw, df_da*da_dx + df_db*db_dx\n",
    "df_dy, df_dz = df_da*da_dy + df_db*db_dy, df_da*da_dz + df_db*db_dz\n",
    "\n",
    "print(f'df/dw at {w}, {x}, {y}, {z} is {df_dw}')\n",
    "print(f'df/dx at {w}, {x}, {y}, {z} is {df_dx}')\n",
    "print(f'df/dy at {w}, {x}, {y}, {z} is {df_dy}')\n",
    "print(f'df/dz at {w}, {x}, {y}, {z} is {df_dz}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "236a7b17",
   "metadata": {
    "origin_pos": 12
   },
   "source": [
    "The fact that we compute derivatives from $f$ back towards the inputs rather than from the inputs forward to the outputs (as we did in the first code snippet above) is what gives this algorithm its name: *backpropagation*.  Note that there are two steps:\n",
    "1. Compute the value of the function, and the single step partials from front to back.  While not done above, this can be combined into a single *forward pass*.\n",
    "2. Compute the gradient of $f$ from back to front.  We call this the *backwards pass*.\n",
    "\n",
    "This is precisely what every deep learning algorithm implements to allow the computation of the gradient of the loss with respect to every weight in the network at one pass.  It is an astonishing fact that we have such a decomposition.\n",
    "\n",
    "To see how to encapsulated this, let's take a quick look at this example.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "16758eb9",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-08-18T19:29:40.399472Z",
     "iopub.status.busy": "2023-08-18T19:29:40.398914Z",
     "iopub.status.idle": "2023-08-18T19:29:40.463500Z",
     "shell.execute_reply": "2023-08-18T19:29:40.461665Z"
    },
    "origin_pos": 14,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "df/dw at -1.0, 0.0, -2.0, 1.0 is -4096.0\n",
      "df/dx at -1.0, 0.0, -2.0, 1.0 is -4096.0\n",
      "df/dy at -1.0, 0.0, -2.0, 1.0 is -4096.0\n",
      "df/dz at -1.0, 0.0, -2.0, 1.0 is -4096.0\n"
     ]
    }
   ],
   "source": [
    "# Initialize as ndarrays, then attach gradients\n",
    "w = torch.tensor([-1.], requires_grad=True)\n",
    "x = torch.tensor([0.], requires_grad=True)\n",
    "y = torch.tensor([-2.], requires_grad=True)\n",
    "z = torch.tensor([1.], requires_grad=True)\n",
    "# Do the computation like usual, tracking gradients\n",
    "a, b = (w + x + y + z)**2, (w + x - y - z)**2\n",
    "u, v = (a + b)**2, (a - b)**2\n",
    "f = (u + v)**2\n",
    "\n",
    "# Execute backward pass\n",
    "f.backward()\n",
    "\n",
    "print(f'df/dw at {w.data.item()}, {x.data.item()}, {y.data.item()}, '\n",
    "      f'{z.data.item()} is {w.grad.data.item()}')\n",
    "print(f'df/dx at {w.data.item()}, {x.data.item()}, {y.data.item()}, '\n",
    "      f'{z.data.item()} is {x.grad.data.item()}')\n",
    "print(f'df/dy at {w.data.item()}, {x.data.item()}, {y.data.item()}, '\n",
    "      f'{z.data.item()} is {y.grad.data.item()}')\n",
    "print(f'df/dz at {w.data.item()}, {x.data.item()}, {y.data.item()}, '\n",
    "      f'{z.data.item()} is {z.grad.data.item()}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2d02454",
   "metadata": {
    "origin_pos": 16
   },
   "source": [
    "All of what we did above can be done automatically by calling `f.backwards()`.\n",
    "\n",
    "\n",
    "## Hessians\n",
    "As with single variable calculus, it is useful to consider higher-order derivatives in order to get a handle on how we can obtain a better approximation to a function than using the gradient alone.\n",
    "\n",
    "There is one immediate problem one encounters when working with higher order derivatives of functions of several variables, and that is there are a large number of them.  If we have a function $f(x_1, \\ldots, x_n)$ of $n$ variables, then we can take $n^{2}$ many second derivatives, namely for any choice of $i$ and $j$:\n",
    "\n",
    "$$\n",
    "\\frac{d^2f}{dx_idx_j} = \\frac{d}{dx_i}\\left(\\frac{d}{dx_j}f\\right).\n",
    "$$\n",
    "\n",
    "This is traditionally assembled into a matrix called the *Hessian*:\n",
    "\n",
    "$$\\mathbf{H}_f = \\begin{bmatrix} \\frac{d^2f}{dx_1dx_1} & \\cdots & \\frac{d^2f}{dx_1dx_n} \\\\ \\vdots & \\ddots & \\vdots \\\\ \\frac{d^2f}{dx_ndx_1} & \\cdots & \\frac{d^2f}{dx_ndx_n} \\\\ \\end{bmatrix}.$$\n",
    ":eqlabel:`eq_hess_def`\n",
    "\n",
    "Not every entry of this matrix is independent.  Indeed, we can show that as long as both *mixed partials* (partial derivatives with respect to more than one variable) exist and are continuous, we can say that for any $i$, and $j$,\n",
    "\n",
    "$$\n",
    "\\frac{d^2f}{dx_idx_j} = \\frac{d^2f}{dx_jdx_i}.\n",
    "$$\n",
    "\n",
    "This follows by considering first perturbing a function in the direction of $x_i$, and then perturbing it in $x_j$ and then comparing the result of that with what happens if we perturb first $x_j$ and then $x_i$, with the knowledge that both of these orders lead to the same final change in the output of $f$.\n",
    "\n",
    "As with single variables, we can use these derivatives to get a far better idea of how the function behaves near a point.  In particular, we can use it to find the best fitting quadratic near a point $\\mathbf{x}_0$, as we saw in a single variable.\n",
    "\n",
    "Let's see an example.  Suppose that $f(x_1, x_2) = a + b_1x_1 + b_2x_2 + c_{11}x_1^{2} + c_{12}x_1x_2 + c_{22}x_2^{2}$.  This is the general form for a quadratic in two variables.  If we look at the value of the function, its gradient, and its Hessian :eqref:`eq_hess_def`, all at the point zero:\n",
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    "f(0,0) & = a, \\\\\n",
    "\\nabla f (0,0) & = \\begin{bmatrix}b_1 \\\\ b_2\\end{bmatrix}, \\\\\n",
    "\\mathbf{H} f (0,0) & = \\begin{bmatrix}2 c_{11} & c_{12} \\\\ c_{12} & 2c_{22}\\end{bmatrix},\n",
    "\\end{aligned}\n",
    "$$\n",
    "\n",
    "we can get our original polynomial back by saying\n",
    "\n",
    "$$\n",
    "f(\\mathbf{x}) = f(0) + \\nabla f (0) \\cdot \\mathbf{x} + \\frac{1}{2}\\mathbf{x}^\\top \\mathbf{H} f (0) \\mathbf{x}.\n",
    "$$\n",
    "\n",
    "In general, if we computed this expansion any point $\\mathbf{x}_0$, we see that\n",
    "\n",
    "$$\n",
    "f(\\mathbf{x}) = f(\\mathbf{x}_0) + \\nabla f (\\mathbf{x}_0) \\cdot (\\mathbf{x}-\\mathbf{x}_0) + \\frac{1}{2}(\\mathbf{x}-\\mathbf{x}_0)^\\top \\mathbf{H} f (\\mathbf{x}_0) (\\mathbf{x}-\\mathbf{x}_0).\n",
    "$$\n",
    "\n",
    "This works for any dimensional input, and provides the best approximating quadratic to any function at a point.  To give an example, let's plot the function\n",
    "\n",
    "$$\n",
    "f(x, y) = xe^{-x^2-y^2}.\n",
    "$$\n",
    "\n",
    "One can compute that the gradient and Hessian are\n",
    "$$\n",
    "\\nabla f(x, y) = e^{-x^2-y^2}\\begin{pmatrix}1-2x^2 \\\\ -2xy\\end{pmatrix} \\; \\textrm{and} \\; \\mathbf{H}f(x, y) = e^{-x^2-y^2}\\begin{pmatrix} 4x^3 - 6x & 4x^2y - 2y \\\\ 4x^2y-2y &4xy^2-2x\\end{pmatrix}.\n",
    "$$\n",
    "\n",
    "And thus, with a little algebra, see that the approximating quadratic at $[-1,0]^\\top$ is\n",
    "\n",
    "$$\n",
    "f(x, y) \\approx e^{-1}\\left(-1 - (x+1) +(x+1)^2+y^2\\right).\n",
    "$$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "b58a17bb",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-08-18T19:29:40.467263Z",
     "iopub.status.busy": "2023-08-18T19:29:40.466658Z",
     "iopub.status.idle": "2023-08-18T19:29:40.756344Z",
     "shell.execute_reply": "2023-08-18T19:29:40.755474Z"
    },
    "origin_pos": 18,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"162.724807pt\" height=\"156.400786pt\" viewBox=\"0 0 162.724807 156.400786\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
       " <metadata>\n",
       "  <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
       "   <cc:Work>\n",
       "    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
       "    <dc:date>2023-08-18T19:29:40.702867</dc:date>\n",
       "    <dc:format>image/svg+xml</dc:format>\n",
       "    <dc:creator>\n",
       "     <cc:Agent>\n",
       "      <dc:title>Matplotlib v3.7.2, https://matplotlib.org/</dc:title>\n",
       "     </cc:Agent>\n",
       "    </dc:creator>\n",
       "   </cc:Work>\n",
       "  </rdf:RDF>\n",
       " </metadata>\n",
       " <defs>\n",
       "  <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 156.400786 \n",
       "L 162.724807 156.400786 \n",
       "L 162.724807 0 \n",
       "L 0 0 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "  </g>\n",
       "  <g id=\"patch_2\">\n",
       "   <path d=\"M 7.2 145.8 \n",
       "L 145.8 145.8 \n",
       "L 145.8 7.2 \n",
       "L 7.2 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "  </g>\n",
       "  <g id=\"pane3d_1\">\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 27.754177 105.476933 \n",
       "L 65.746483 73.471623 \n",
       "L 65.294849 27.015534 \n",
       "L 25.804101 56.674423 \n",
       "\" style=\"fill: #f2f2f2; opacity: 0.5; stroke: #f2f2f2; stroke-linejoin: miter\"/>\n",
       "   </g>\n",
       "  </g>\n",
       "  <g id=\"pane3d_2\">\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 65.746483 73.471623 \n",
       "L 127.452778 91.388092 \n",
       "L 129.283933 43.59906 \n",
       "L 65.294849 27.015534 \n",
       "\" style=\"fill: #e6e6e6; opacity: 0.5; stroke: #e6e6e6; stroke-linejoin: miter\"/>\n",
       "   </g>\n",
       "  </g>\n",
       "  <g id=\"pane3d_3\">\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 27.754177 105.476933 \n",
       "L 92.543212 126.21441 \n",
       "L 127.452778 91.388092 \n",
       "L 65.746483 73.471623 \n",
       "\" style=\"fill: #ececec; opacity: 0.5; stroke: #ececec; stroke-linejoin: miter\"/>\n",
       "   </g>\n",
       "  </g>\n",
       "  <g id=\"axis3d_1\">\n",
       "   <g id=\"line2d_1\">\n",
       "    <path d=\"M 27.754177 105.476933 \n",
       "L 92.543212 126.21441 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"text_1\">\n",
       "    <!-- x -->\n",
       "    <g transform=\"translate(42.940071 147.693651) scale(0.1 -0.1)\">\n",
       "     <defs>\n",
       "      <path id=\"DejaVuSans-78\" d=\"M 3513 3500 \n",
       "L 2247 1797 \n",
       "L 3578 0 \n",
       "L 2900 0 \n",
       "L 1881 1375 \n",
       "L 863 0 \n",
       "L 184 0 \n",
       "L 1544 1831 \n",
       "L 300 3500 \n",
       "L 978 3500 \n",
       "L 1906 2253 \n",
       "L 2834 3500 \n",
       "L 3513 3500 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "     </defs>\n",
       "     <use xlink:href=\"#DejaVuSans-78\"/>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"Line3DCollection_1\">\n",
       "    <path d=\"M 28.995517 105.874257 \n",
       "L 66.932342 73.815938 \n",
       "L 66.522787 27.333769 \n",
       "\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8\"/>\n",
       "    <path d=\"M 44.059364 110.695848 \n",
       "L 81.311741 77.991008 \n",
       "L 81.417999 31.194039 \n",
       "\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8\"/>\n",
       "    <path d=\"M 59.439711 115.618742 \n",
       "L 95.971967 82.247615 \n",
       "L 96.614736 35.132452 \n",
       "\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8\"/>\n",
       "    <path d=\"M 75.146636 120.646167 \n",
       "L 110.921328 86.588173 \n",
       "L 112.122247 39.151407 \n",
       "\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8\"/>\n",
       "    <path d=\"M 91.190654 125.781488 \n",
       "L 126.168463 91.01519 \n",
       "L 127.950165 43.253399 \n",
       "\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8\"/>\n",
       "   </g>\n",
       "   <g id=\"xtick_1\">\n",
       "    <g id=\"line2d_2\">\n",
       "     <path d=\"M 29.321354 105.598911 \n",
       "L 28.34268 106.425934 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "    </g>\n",
       "    <g id=\"text_2\">\n",
       "     <!-- −2 -->\n",
       "     <g transform=\"translate(15.3872 126.526523) scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-2212\" d=\"M 678 2272 \n",
       "L 4684 2272 \n",
       "L 4684 1741 \n",
       "L 678 1741 \n",
       "L 678 2272 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
       "L 3431 531 \n",
       "L 3431 0 \n",
       "L 469 0 \n",
       "L 469 531 \n",
       "Q 828 903 1448 1529 \n",
       "Q 2069 2156 2228 2338 \n",
       "Q 2531 2678 2651 2914 \n",
       "Q 2772 3150 2772 3378 \n",
       "Q 2772 3750 2511 3984 \n",
       "Q 2250 4219 1831 4219 \n",
       "Q 1534 4219 1204 4116 \n",
       "Q 875 4013 500 3803 \n",
       "L 500 4441 \n",
       "Q 881 4594 1212 4672 \n",
       "Q 1544 4750 1819 4750 \n",
       "Q 2544 4750 2975 4387 \n",
       "Q 3406 4025 3406 3419 \n",
       "Q 3406 3131 3298 2873 \n",
       "Q 3191 2616 2906 2266 \n",
       "Q 2828 2175 2409 1742 \n",
       "Q 1991 1309 1228 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-2212\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-32\" x=\"83.789062\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"xtick_2\">\n",
       "    <g id=\"line2d_3\">\n",
       "     <path d=\"M 44.379551 110.414748 \n",
       "L 43.417837 111.259061 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "    </g>\n",
       "    <g id=\"text_3\">\n",
       "     <!-- −1 -->\n",
       "     <g transform=\"translate(30.476381 131.513626) scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
       "L 1825 531 \n",
       "L 1825 4091 \n",
       "L 703 3866 \n",
       "L 703 4441 \n",
       "L 1819 4666 \n",
       "L 2450 4666 \n",
       "L 2450 531 \n",
       "L 3481 531 \n",
       "L 3481 0 \n",
       "L 794 0 \n",
       "L 794 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-2212\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-31\" x=\"83.789062\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"xtick_3\">\n",
       "    <g id=\"line2d_4\">\n",
       "     <path d=\"M 59.753936 115.331707 \n",
       "L 58.810116 116.193859 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "    </g>\n",
       "    <g id=\"text_4\">\n",
       "     <!-- 0 -->\n",
       "     <g transform=\"translate(50.074362 136.606146) scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
       "Q 1547 4250 1301 3770 \n",
       "Q 1056 3291 1056 2328 \n",
       "Q 1056 1369 1301 889 \n",
       "Q 1547 409 2034 409 \n",
       "Q 2525 409 2770 889 \n",
       "Q 3016 1369 3016 2328 \n",
       "Q 3016 3291 2770 3770 \n",
       "Q 2525 4250 2034 4250 \n",
       "z\n",
       "M 2034 4750 \n",
       "Q 2819 4750 3233 4129 \n",
       "Q 3647 3509 3647 2328 \n",
       "Q 3647 1150 3233 529 \n",
       "Q 2819 -91 2034 -91 \n",
       "Q 1250 -91 836 529 \n",
       "Q 422 1150 422 2328 \n",
       "Q 422 3509 836 4129 \n",
       "Q 1250 4750 2034 4750 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"xtick_4\">\n",
       "    <g id=\"line2d_5\">\n",
       "     <path d=\"M 75.454573 120.353007 \n",
       "L 74.529628 121.233568 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "    </g>\n",
       "    <g id=\"text_5\">\n",
       "     <!-- 1 -->\n",
       "     <g transform=\"translate(65.811678 141.807462) scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"xtick_5\">\n",
       "    <g id=\"line2d_6\">\n",
       "     <path d=\"M 91.491959 125.482004 \n",
       "L 90.58692 126.38157 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "    </g>\n",
       "    <g id=\"text_6\">\n",
       "     <!-- 2 -->\n",
       "     <g transform=\"translate(81.888835 147.121099) scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       "  <g id=\"axis3d_2\">\n",
       "   <g id=\"line2d_7\">\n",
       "    <path d=\"M 127.452778 91.388092 \n",
       "L 92.543212 126.21441 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"text_7\">\n",
       "    <!-- y -->\n",
       "    <g transform=\"translate(130.155124 134.245064) scale(0.1 -0.1)\">\n",
       "     <defs>\n",
       "      <path id=\"DejaVuSans-79\" d=\"M 2059 -325 \n",
       "Q 1816 -950 1584 -1140 \n",
       "Q 1353 -1331 966 -1331 \n",
       "L 506 -1331 \n",
       "L 506 -850 \n",
       "L 844 -850 \n",
       "Q 1081 -850 1212 -737 \n",
       "Q 1344 -625 1503 -206 \n",
       "L 1606 56 \n",
       "L 191 3500 \n",
       "L 800 3500 \n",
       "L 1894 763 \n",
       "L 2988 3500 \n",
       "L 3597 3500 \n",
       "L 2059 -325 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "     </defs>\n",
       "     <use xlink:href=\"#DejaVuSans-79\"/>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"Line3DCollection_2\">\n",
       "    <path d=\"M 26.653462 56.036525 \n",
       "L 28.569187 104.790355 \n",
       "L 93.294356 125.465057 \n",
       "\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8\"/>\n",
       "    <path d=\"M 36.645621 48.532074 \n",
       "L 38.16423 96.707342 \n",
       "L 102.130027 116.650456 \n",
       "\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8\"/>\n",
       "    <path d=\"M 46.281942 41.294871 \n",
       "L 47.429799 88.901883 \n",
       "L 110.649228 108.151571 \n",
       "\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8\"/>\n",
       "    <path d=\"M 55.581099 34.310889 \n",
       "L 56.382576 81.359924 \n",
       "L 118.868662 99.951737 \n",
       "\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8\"/>\n",
       "    <path d=\"M 64.560484 27.567067 \n",
       "L 65.038138 74.068343 \n",
       "L 126.803878 92.035445 \n",
       "\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8\"/>\n",
       "   </g>\n",
       "   <g id=\"xtick_6\">\n",
       "    <g id=\"line2d_8\">\n",
       "     <path d=\"M 92.753612 125.292331 \n",
       "L 94.377006 125.81088 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "    </g>\n",
       "    <g id=\"text_8\">\n",
       "     <!-- −2 -->\n",
       "     <g transform=\"translate(96.613394 144.200802) scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-2212\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-32\" x=\"83.789062\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"xtick_7\">\n",
       "    <g id=\"line2d_9\">\n",
       "     <path d=\"M 101.59604 116.483971 \n",
       "L 103.199126 116.983777 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "    </g>\n",
       "    <g id=\"text_9\">\n",
       "     <!-- −1 -->\n",
       "     <g transform=\"translate(105.262226 135.174659) scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-2212\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-31\" x=\"83.789062\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"xtick_8\">\n",
       "    <g id=\"line2d_10\">\n",
       "     <path d=\"M 110.121866 107.990994 \n",
       "L 111.705044 108.473056 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "    </g>\n",
       "    <g id=\"text_10\">\n",
       "     <!-- 0 -->\n",
       "     <g transform=\"translate(117.790838 126.472106) scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"xtick_9\">\n",
       "    <g id=\"line2d_11\">\n",
       "     <path d=\"M 118.347793 99.79676 \n",
       "L 119.91146 100.262007 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "    </g>\n",
       "    <g id=\"text_11\">\n",
       "     <!-- 1 -->\n",
       "     <g transform=\"translate(125.835923 118.076048) scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"xtick_10\">\n",
       "    <g id=\"line2d_12\">\n",
       "     <path d=\"M 126.289371 91.885779 \n",
       "L 127.833918 92.335074 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "    </g>\n",
       "    <g id=\"text_12\">\n",
       "     <!-- 2 -->\n",
       "     <g transform=\"translate(133.602571 109.970574) scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       "  <g id=\"axis3d_3\">\n",
       "   <g id=\"line2d_13\">\n",
       "    <path d=\"M 127.452778 91.388092 \n",
       "L 129.283933 43.59906 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"Line3DCollection_3\">\n",
       "    <path d=\"M 127.488109 90.466025 \n",
       "L 65.737756 72.573967 \n",
       "L 27.716594 104.536392 \n",
       "\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8\"/>\n",
       "    <path d=\"M 127.916087 79.296776 \n",
       "L 65.632086 61.704502 \n",
       "L 27.261211 93.139986 \n",
       "\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8\"/>\n",
       "    <path d=\"M 128.351588 67.931162 \n",
       "L 65.524633 50.651699 \n",
       "L 26.797565 81.536821 \n",
       "\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8\"/>\n",
       "    <path d=\"M 128.794814 56.363961 \n",
       "L 65.415353 39.410878 \n",
       "L 26.325432 69.721221 \n",
       "\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8\"/>\n",
       "    <path d=\"M 129.245972 44.589762 \n",
       "L 65.304198 27.977202 \n",
       "L 25.844574 57.687295 \n",
       "\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8\"/>\n",
       "   </g>\n",
       "   <g id=\"xtick_11\">\n",
       "    <g id=\"line2d_14\">\n",
       "     <path d=\"M 126.973746 90.31699 \n",
       "L 128.517863 90.764395 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "    </g>\n",
       "    <g id=\"text_13\">\n",
       "     <!-- −1.0 -->\n",
       "     <g transform=\"translate(130.683112 95.901632) scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-2e\" d=\"M 684 794 \n",
       "L 1344 794 \n",
       "L 1344 0 \n",
       "L 684 0 \n",
       "L 684 794 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-2212\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-31\" x=\"83.789062\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-2e\" x=\"147.412109\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-30\" x=\"179.199219\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"xtick_12\">\n",
       "    <g id=\"line2d_15\">\n",
       "     <path d=\"M 127.397099 79.150186 \n",
       "L 128.955107 79.590249 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "    </g>\n",
       "    <g id=\"text_14\">\n",
       "     <!-- −0.5 -->\n",
       "     <g transform=\"translate(131.241994 84.767971) scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-35\" d=\"M 691 4666 \n",
       "L 3169 4666 \n",
       "L 3169 4134 \n",
       "L 1269 4134 \n",
       "L 1269 2991 \n",
       "Q 1406 3038 1543 3061 \n",
       "Q 1681 3084 1819 3084 \n",
       "Q 2600 3084 3056 2656 \n",
       "Q 3513 2228 3513 1497 \n",
       "Q 3513 744 3044 326 \n",
       "Q 2575 -91 1722 -91 \n",
       "Q 1428 -91 1123 -41 \n",
       "Q 819 9 494 109 \n",
       "L 494 744 \n",
       "Q 775 591 1075 516 \n",
       "Q 1375 441 1709 441 \n",
       "Q 2250 441 2565 725 \n",
       "Q 2881 1009 2881 1497 \n",
       "Q 2881 1984 2565 2268 \n",
       "Q 2250 2553 1709 2553 \n",
       "Q 1456 2553 1204 2497 \n",
       "Q 953 2441 691 2322 \n",
       "L 691 4666 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-2212\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-30\" x=\"83.789062\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-2e\" x=\"147.412109\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-35\" x=\"179.199219\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"xtick_13\">\n",
       "    <g id=\"line2d_16\">\n",
       "     <path d=\"M 127.827893 67.787129 \n",
       "L 129.400043 68.219521 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "    </g>\n",
       "    <g id=\"text_15\">\n",
       "     <!-- 0.0 -->\n",
       "     <g transform=\"translate(136.000498 73.439533) scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"xtick_14\">\n",
       "    <g id=\"line2d_17\">\n",
       "     <path d=\"M 128.266325 56.222598 \n",
       "L 129.852876 56.646977 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "    </g>\n",
       "    <g id=\"text_16\">\n",
       "     <!-- 0.5 -->\n",
       "     <g transform=\"translate(136.579194 61.911161) scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-35\" x=\"95.410156\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"xtick_15\">\n",
       "    <g id=\"line2d_18\">\n",
       "     <path d=\"M 128.7126 44.451188 \n",
       "L 130.313818 44.867196 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "    </g>\n",
       "    <g id=\"text_17\">\n",
       "     <!-- 1.0 -->\n",
       "     <g transform=\"translate(137.168195 50.177516) scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"Line3DCollection_4\">\n",
       "    <path d=\"M 28.892801 81.271898 \n",
       "L 29.290719 80.956312 \n",
       "L 29.688075 80.641578 \n",
       "L 30.084869 80.327739 \n",
       "L 30.481105 80.01484 \n",
       "L 30.876786 79.70293 \n",
       "L 31.271914 79.392061 \n",
       "L 31.666492 79.082285 \n",
       "L 32.060522 78.773655 \n",
       "L 32.454008 78.466227 \n",
       "L 32.84695 78.160058 \n",
       "L 33.239353 77.855202 \n",
       "L 33.631219 77.551714 \n",
       "L 34.022549 77.249646 \n",
       "L 34.413345 76.949047 \n",
       "L 34.803611 76.649961 \n",
       "L 35.193345 76.35243 \n",
       "L 35.582551 76.056486 \n",
       "L 35.971231 75.762155 \n",
       "L 36.359382 75.469454 \n",
       "L 36.747008 75.178391 \n",
       "L 37.134107 74.888959 \n",
       "L 37.520679 74.601144 \n",
       "L 37.906723 74.314916 \n",
       "L 38.29224 74.030228 \n",
       "L 38.677225 73.747025 \n",
       "L 39.061678 73.465231 \n",
       "L 39.445596 73.184754 \n",
       "L 39.828978 72.905491 \n",
       "L 40.211818 72.627319 \n",
       "L 40.594116 72.350101 \n",
       "L 40.975865 72.073686 \n",
       "L 41.357064 71.797908 \n",
       "L 41.737708 71.522592 \n",
       "L 42.117792 71.247549 \n",
       "L 42.497314 70.972583 \n",
       "L 42.876268 70.697491 \n",
       "L 43.254651 70.422065 \n",
       "L 43.63246 70.146096 \n",
       "L 44.00969 69.869375 \n",
       "L 44.386339 69.591697 \n",
       "L 44.762404 69.312864 \n",
       "L 45.137882 69.032685 \n",
       "L 45.512773 68.750982 \n",
       "L 45.887074 68.467593 \n",
       "L 46.260785 68.182369 \n",
       "L 46.633906 67.895184 \n",
       "L 47.006438 67.60593 \n",
       "L 47.378382 67.314523 \n",
       "L 47.749741 67.020903 \n",
       "L 48.120516 66.725034 \n",
       "L 48.490711 66.426908 \n",
       "L 48.860329 66.126539 \n",
       "L 49.229375 65.823968 \n",
       "L 49.597853 65.519261 \n",
       "L 49.96577 65.212508 \n",
       "L 50.33313 64.90382 \n",
       "L 50.699939 64.593328 \n",
       "L 51.066204 64.281184 \n",
       "L 51.431931 63.967553 \n",
       "L 51.797125 63.652616 \n",
       "L 52.161794 63.336563 \n",
       "L 52.525944 63.019594 \n",
       "L 52.88958 62.701913 \n",
       "L 53.252709 62.383728 \n",
       "L 53.615336 62.065247 \n",
       "L 53.977467 61.746673 \n",
       "L 54.339107 61.428207 \n",
       "L 54.700261 61.11004 \n",
       "L 55.060933 60.792357 \n",
       "L 55.421126 60.475328 \n",
       "L 55.780845 60.159113 \n",
       "L 56.140092 59.843857 \n",
       "L 56.498871 59.529691 \n",
       "L 56.857184 59.216731 \n",
       "L 57.215032 58.905075 \n",
       "L 57.572416 58.594808 \n",
       "L 57.92934 58.285997 \n",
       "L 58.285803 57.978696 \n",
       "L 58.641805 57.672942 \n",
       "L 58.997348 57.368759 \n",
       "L 59.352431 57.066159 \n",
       "L 59.707054 56.765141 \n",
       "L 60.061219 56.465691 \n",
       "L 60.414922 56.167792 \n",
       "L 60.768165 55.87141 \n",
       "L 61.120948 55.576509 \n",
       "L 61.473269 55.283047 \n",
       "L 61.825129 54.990975 \n",
       "L 62.176527 54.700241 \n",
       "L 62.527461 54.41079 \n",
       "L 62.877932 54.122566 \n",
       "L 63.227941 53.835512 \n",
       "L 63.577485 53.54957 \n",
       "L 63.926565 53.264684 \n",
       "L 64.275181 52.980797 \n",
       "L 64.623334 52.697855 \n",
       "L 64.971021 52.415808 \n",
       "L 65.318244 52.134605 \n",
       "L 65.665004 51.854199 \n",
       "L 66.011299 51.574547 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 34.986226 83.155976 \n",
       "L 35.381571 82.844118 \n",
       "L 35.77638 82.53405 \n",
       "L 36.170657 82.22592 \n",
       "L 36.564407 81.919885 \n",
       "L 36.957637 81.616112 \n",
       "L 37.350349 81.314775 \n",
       "L 37.74255 81.016056 \n",
       "L 38.134245 80.720139 \n",
       "L 38.52544 80.427214 \n",
       "L 38.916137 80.137473 \n",
       "L 39.306342 79.851104 \n",
       "L 39.696061 79.568293 \n",
       "L 40.085294 79.289219 \n",
       "L 40.474047 79.014051 \n",
       "L 40.862322 78.742942 \n",
       "L 41.25012 78.476029 \n",
       "L 41.637443 78.213427 \n",
       "L 42.024293 77.955224 \n",
       "L 42.410665 77.70148 \n",
       "L 42.79656 77.452219 \n",
       "L 43.181977 77.207428 \n",
       "L 43.566909 76.967053 \n",
       "L 43.951352 76.730994 \n",
       "L 44.335302 76.499102 \n",
       "L 44.718748 76.271181 \n",
       "L 45.101685 76.046979 \n",
       "L 45.484103 75.826193 \n",
       "L 45.865991 75.608467 \n",
       "L 46.247337 75.39339 \n",
       "L 46.628132 75.180501 \n",
       "L 47.00836 74.969291 \n",
       "L 47.388011 74.759204 \n",
       "L 47.76707 74.549643 \n",
       "L 48.145523 74.339978 \n",
       "L 48.523359 74.129546 \n",
       "L 48.900562 73.917663 \n",
       "L 49.277121 73.703632 \n",
       "L 49.653024 73.486749 \n",
       "L 50.028259 73.266312 \n",
       "L 50.402816 73.041631 \n",
       "L 50.776687 72.812038 \n",
       "L 51.149865 72.576897 \n",
       "L 51.522342 72.335609 \n",
       "L 51.894116 72.087625 \n",
       "L 52.265183 71.832454 \n",
       "L 52.635544 71.569665 \n",
       "L 53.005198 71.298903 \n",
       "L 53.374149 71.019886 \n",
       "L 53.742403 70.732413 \n",
       "L 54.109966 70.436367 \n",
       "L 54.476844 70.131718 \n",
       "L 54.84305 69.81852 \n",
       "L 55.208593 69.496913 \n",
       "L 55.573487 69.167121 \n",
       "L 55.937745 68.829444 \n",
       "L 56.301381 68.484262 \n",
       "L 56.664411 68.132021 \n",
       "L 57.02685 67.773226 \n",
       "L 57.388714 67.408443 \n",
       "L 57.75002 67.03828 \n",
       "L 58.110782 66.663382 \n",
       "L 58.471018 66.284422 \n",
       "L 58.830741 65.902093 \n",
       "L 59.189966 65.517097 \n",
       "L 59.548707 65.130135 \n",
       "L 59.906976 64.741899 \n",
       "L 60.264785 64.353067 \n",
       "L 60.622145 63.964289 \n",
       "L 60.979065 63.576186 \n",
       "L 61.335553 63.189341 \n",
       "L 61.691617 62.804294 \n",
       "L 62.047262 62.421541 \n",
       "L 62.402493 62.041523 \n",
       "L 62.757314 61.664635 \n",
       "L 63.111726 61.291217 \n",
       "L 63.465733 60.921553 \n",
       "L 63.819335 60.555875 \n",
       "L 64.172531 60.194367 \n",
       "L 64.52532 59.83716 \n",
       "L 64.877703 59.484336 \n",
       "L 65.229676 59.135938 \n",
       "L 65.581236 58.791964 \n",
       "L 65.932383 58.452377 \n",
       "L 66.283112 58.117108 \n",
       "L 66.633419 57.786058 \n",
       "L 66.983304 57.459103 \n",
       "L 67.332761 57.136099 \n",
       "L 67.681787 56.816887 \n",
       "L 68.030381 56.50129 \n",
       "L 68.378537 56.189129 \n",
       "L 68.726254 55.880214 \n",
       "L 69.07353 55.574351 \n",
       "L 69.420359 55.27135 \n",
       "L 69.766743 54.97102 \n",
       "L 70.112677 54.673174 \n",
       "L 70.458161 54.377631 \n",
       "L 70.803192 54.084218 \n",
       "L 71.14777 53.792769 \n",
       "L 71.491893 53.503127 \n",
       "L 71.83556 53.215146 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 41.131668 85.087564 \n",
       "L 41.524465 84.784859 \n",
       "L 41.916764 84.485674 \n",
       "L 42.30857 84.190354 \n",
       "L 42.699893 83.899262 \n",
       "L 43.090742 83.612781 \n",
       "L 43.481123 83.331316 \n",
       "L 43.871047 83.055283 \n",
       "L 44.26052 82.785112 \n",
       "L 44.649552 82.521239 \n",
       "L 45.038148 82.264105 \n",
       "L 45.426316 82.014147 \n",
       "L 45.814062 81.771791 \n",
       "L 46.201391 81.53745 \n",
       "L 46.588306 81.311511 \n",
       "L 46.974813 81.094327 \n",
       "L 47.360909 80.886214 \n",
       "L 47.746596 80.687433 \n",
       "L 48.131873 80.498188 \n",
       "L 48.516735 80.318613 \n",
       "L 48.901176 80.148763 \n",
       "L 49.28519 79.988605 \n",
       "L 49.668766 79.838011 \n",
       "L 50.051892 79.696749 \n",
       "L 50.434557 79.564475 \n",
       "L 50.81674 79.440733 \n",
       "L 51.198427 79.324942 \n",
       "L 51.579598 79.216403 \n",
       "L 51.960231 79.114293 \n",
       "L 52.340302 79.017668 \n",
       "L 52.719789 78.925467 \n",
       "L 53.098666 78.836518 \n",
       "L 53.476909 78.749543 \n",
       "L 53.85449 78.663174 \n",
       "L 54.231384 78.575962 \n",
       "L 54.607567 78.486388 \n",
       "L 54.983013 78.392887 \n",
       "L 55.357701 78.293861 \n",
       "L 55.731607 78.187698 \n",
       "L 56.104713 78.072794 \n",
       "L 56.477002 77.947573 \n",
       "L 56.848458 77.810511 \n",
       "L 57.219071 77.66015 \n",
       "L 57.588831 77.495129 \n",
       "L 57.957733 77.314194 \n",
       "L 58.325775 77.116223 \n",
       "L 58.692959 76.900241 \n",
       "L 59.059289 76.665433 \n",
       "L 59.424774 76.41116 \n",
       "L 59.789426 76.136965 \n",
       "L 60.15326 75.842582 \n",
       "L 60.516292 75.527943 \n",
       "L 60.878544 75.193173 \n",
       "L 61.240037 74.838592 \n",
       "L 61.600798 74.46471 \n",
       "L 61.960851 74.072218 \n",
       "L 62.320225 73.661976 \n",
       "L 62.678949 73.235005 \n",
       "L 63.037051 72.792464 \n",
       "L 63.39456 72.335639 \n",
       "L 63.751506 71.865921 \n",
       "L 64.107915 71.384787 \n",
       "L 64.463816 70.893777 \n",
       "L 64.819235 70.394475 \n",
       "L 65.174195 69.88849 \n",
       "L 65.528718 69.37743 \n",
       "L 65.882827 68.862881 \n",
       "L 66.236537 68.346398 \n",
       "L 66.589867 67.829476 \n",
       "L 66.94283 67.313541 \n",
       "L 67.295437 66.799935 \n",
       "L 67.647699 66.289903 \n",
       "L 67.999621 65.784583 \n",
       "L 68.351209 65.285 \n",
       "L 68.702466 64.79206 \n",
       "L 69.053394 64.306551 \n",
       "L 69.403991 63.829132 \n",
       "L 69.754257 63.360344 \n",
       "L 70.104187 62.900611 \n",
       "L 70.453777 62.45024 \n",
       "L 70.803023 62.009427 \n",
       "L 71.151918 61.578274 \n",
       "L 71.500454 61.156782 \n",
       "L 71.848627 60.744869 \n",
       "L 72.196427 60.34238 \n",
       "L 72.543847 59.94909 \n",
       "L 72.890882 59.564719 \n",
       "L 73.237522 59.188938 \n",
       "L 73.583761 58.82138 \n",
       "L 73.929593 58.461647 \n",
       "L 74.27501 58.109322 \n",
       "L 74.620007 57.763972 \n",
       "L 74.96458 57.425154 \n",
       "L 75.308722 57.092431 \n",
       "L 75.652429 56.765362 \n",
       "L 75.995697 56.443521 \n",
       "L 76.338524 56.126492 \n",
       "L 76.680905 55.813876 \n",
       "L 77.022837 55.505293 \n",
       "L 77.36432 55.200382 \n",
       "L 77.705351 54.898807 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 47.328316 87.025338 \n",
       "L 47.718353 86.730105 \n",
       "L 48.107909 86.439886 \n",
       "L 48.496988 86.15519 \n",
       "L 48.885602 85.876559 \n",
       "L 49.273762 85.604565 \n",
       "L 49.661472 85.339806 \n",
       "L 50.048745 85.082904 \n",
       "L 50.435588 84.834496 \n",
       "L 50.822009 84.595231 \n",
       "L 51.208015 84.365765 \n",
       "L 51.593613 84.146746 \n",
       "L 51.978809 83.938807 \n",
       "L 52.363605 83.742561 \n",
       "L 52.748005 83.55858 \n",
       "L 53.132012 83.387389 \n",
       "L 53.515621 83.229454 \n",
       "L 53.898832 83.085161 \n",
       "L 54.281642 82.95481 \n",
       "L 54.66404 82.838596 \n",
       "L 55.046019 82.736598 \n",
       "L 55.427569 82.648762 \n",
       "L 55.808672 82.574895 \n",
       "L 56.189314 82.514646 \n",
       "L 56.569477 82.467501 \n",
       "L 56.949138 82.432775 \n",
       "L 57.328275 82.409606 \n",
       "L 57.706864 82.396947 \n",
       "L 58.084877 82.393574 \n",
       "L 58.462287 82.398083 \n",
       "L 58.839066 82.408895 \n",
       "L 59.215183 82.424269 \n",
       "L 59.59061 82.442309 \n",
       "L 59.965318 82.460983 \n",
       "L 60.339276 82.478144 \n",
       "L 60.71246 82.491543 \n",
       "L 61.084841 82.49886 \n",
       "L 61.456398 82.497731 \n",
       "L 61.827109 82.485772 \n",
       "L 62.196955 82.460612 \n",
       "L 62.56592 82.419922 \n",
       "L 62.933994 82.361449 \n",
       "L 63.301169 82.283045 \n",
       "L 63.66744 82.182699 \n",
       "L 64.032808 82.058562 \n",
       "L 64.397276 81.908981 \n",
       "L 64.760853 81.732517 \n",
       "L 65.123551 81.527972 \n",
       "L 65.485386 81.294401 \n",
       "L 65.846378 81.031132 \n",
       "L 66.20655 80.737777 \n",
       "L 66.565927 80.41423 \n",
       "L 66.924538 80.060681 \n",
       "L 67.282414 79.677599 \n",
       "L 67.639586 79.265736 \n",
       "L 67.996089 78.826111 \n",
       "L 68.351957 78.359995 \n",
       "L 68.707224 77.868889 \n",
       "L 69.061924 77.354505 \n",
       "L 69.416091 76.818742 \n",
       "L 69.769758 76.263652 \n",
       "L 70.122954 75.691416 \n",
       "L 70.47571 75.104312 \n",
       "L 70.828052 74.50468 \n",
       "L 71.180005 73.8949 \n",
       "L 71.531589 73.27735 \n",
       "L 71.882826 72.654382 \n",
       "L 72.233729 72.028297 \n",
       "L 72.584313 71.401314 \n",
       "L 72.934589 70.775547 \n",
       "L 73.284562 70.152989 \n",
       "L 73.634241 69.535487 \n",
       "L 73.983625 68.924737 \n",
       "L 74.332716 68.322262 \n",
       "L 74.681511 67.729414 \n",
       "L 75.030007 67.147362 \n",
       "L 75.378198 66.577096 \n",
       "L 75.726078 66.019422 \n",
       "L 76.073637 65.474976 \n",
       "L 76.420868 64.944217 \n",
       "L 76.767761 64.427446 \n",
       "L 77.114305 63.924815 \n",
       "L 77.46049 63.436334 \n",
       "L 77.806308 62.961883 \n",
       "L 78.151746 62.501237 \n",
       "L 78.496794 62.054066 \n",
       "L 78.841446 61.619953 \n",
       "L 79.185689 61.198417 \n",
       "L 79.529517 60.788913 \n",
       "L 79.872923 60.390852 \n",
       "L 80.215898 60.003616 \n",
       "L 80.558437 59.626561 \n",
       "L 80.900535 59.259034 \n",
       "L 81.242185 58.900382 \n",
       "L 81.583384 58.549954 \n",
       "L 81.924129 58.207115 \n",
       "L 82.264417 57.871247 \n",
       "L 82.604245 57.541759 \n",
       "L 82.943612 57.218086 \n",
       "L 83.282517 56.899692 \n",
       "L 83.620957 56.586077 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 53.575324 88.884744 \n",
       "L 53.962161 88.580812 \n",
       "L 54.348484 88.281038 \n",
       "L 54.734295 87.985838 \n",
       "L 55.119601 87.69565 \n",
       "L 55.504409 87.410936 \n",
       "L 55.888723 87.132183 \n",
       "L 56.272547 86.859893 \n",
       "L 56.655887 86.594583 \n",
       "L 57.038748 86.336781 \n",
       "L 57.421132 86.087016 \n",
       "L 57.803042 85.845815 \n",
       "L 58.184483 85.613692 \n",
       "L 58.565452 85.391144 \n",
       "L 58.945952 85.178636 \n",
       "L 59.325982 84.976593 \n",
       "L 59.705538 84.785395 \n",
       "L 60.084619 84.605356 \n",
       "L 60.463221 84.43672 \n",
       "L 60.841335 84.279648 \n",
       "L 61.218955 84.134205 \n",
       "L 61.596073 84.000349 \n",
       "L 61.972677 83.877924 \n",
       "L 62.348756 83.766651 \n",
       "L 62.724299 83.666112 \n",
       "L 63.099289 83.575755 \n",
       "L 63.473712 83.494882 \n",
       "L 63.847554 83.422647 \n",
       "L 64.220796 83.358059 \n",
       "L 64.593422 83.299978 \n",
       "L 64.965416 83.247127 \n",
       "L 65.336759 83.198092 \n",
       "L 65.707437 83.151337 \n",
       "L 66.077434 83.105211 \n",
       "L 66.446734 83.057971 \n",
       "L 66.815324 83.007792 \n",
       "L 67.183192 82.952788 \n",
       "L 67.550329 82.891038 \n",
       "L 67.916726 82.820604 \n",
       "L 68.282378 82.739556 \n",
       "L 68.64728 82.646003 \n",
       "L 69.011433 82.538109 \n",
       "L 69.374838 82.414128 \n",
       "L 69.7375 82.272419 \n",
       "L 70.099425 82.111481 \n",
       "L 70.460624 81.929965 \n",
       "L 70.821108 81.726699 \n",
       "L 71.180892 81.500707 \n",
       "L 71.539993 81.25122 \n",
       "L 71.898429 80.977691 \n",
       "L 72.256221 80.679801 \n",
       "L 72.613388 80.357467 \n",
       "L 72.969955 80.010841 \n",
       "L 73.325944 79.640306 \n",
       "L 73.681379 79.246476 \n",
       "L 74.036282 78.830178 \n",
       "L 74.390677 78.392448 \n",
       "L 74.744587 77.93451 \n",
       "L 75.098032 77.457758 \n",
       "L 75.451032 76.963736 \n",
       "L 75.803607 76.454116 \n",
       "L 76.155773 75.930671 \n",
       "L 76.507545 75.395255 \n",
       "L 76.858937 74.849772 \n",
       "L 77.209959 74.296153 \n",
       "L 77.560619 73.736335 \n",
       "L 77.910926 73.172227 \n",
       "L 78.260882 72.605699 \n",
       "L 78.610492 72.03855 \n",
       "L 78.959755 71.472497 \n",
       "L 79.30867 70.909158 \n",
       "L 79.657236 70.350029 \n",
       "L 80.005447 69.796486 \n",
       "L 80.353299 69.249762 \n",
       "L 80.700784 68.710951 \n",
       "L 81.047896 68.181002 \n",
       "L 81.394626 67.660712 \n",
       "L 81.740968 67.150735 \n",
       "L 82.08691 66.651581 \n",
       "L 82.432446 66.163623 \n",
       "L 82.777567 65.687101 \n",
       "L 83.122263 65.222136 \n",
       "L 83.466527 64.768734 \n",
       "L 83.810352 64.326796 \n",
       "L 84.15373 63.896137 \n",
       "L 84.496654 63.476487 \n",
       "L 84.839121 63.067509 \n",
       "L 85.181122 62.668809 \n",
       "L 85.522655 62.279946 \n",
       "L 85.863717 61.900439 \n",
       "L 86.204302 61.529789 \n",
       "L 86.544409 61.167471 \n",
       "L 86.884037 60.812956 \n",
       "L 87.223183 60.465714 \n",
       "L 87.561848 60.125216 \n",
       "L 87.900031 59.790949 \n",
       "L 88.237733 59.46241 \n",
       "L 88.574953 59.139121 \n",
       "L 88.911692 58.820622 \n",
       "L 89.247953 58.50648 \n",
       "L 89.583735 58.196289 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 59.874475 90.647197 \n",
       "L 60.257863 90.315173 \n",
       "L 60.640686 89.983639 \n",
       "L 61.022941 89.652597 \n",
       "L 61.404631 89.322043 \n",
       "L 61.78576 88.991976 \n",
       "L 62.166325 88.662397 \n",
       "L 62.546329 88.333304 \n",
       "L 62.925773 88.004695 \n",
       "L 63.30466 87.676569 \n",
       "L 63.682989 87.348927 \n",
       "L 64.060762 87.021766 \n",
       "L 64.437982 86.695084 \n",
       "L 64.814646 86.368883 \n",
       "L 65.190759 86.04316 \n",
       "L 65.566321 85.717914 \n",
       "L 65.941332 85.393144 \n",
       "L 66.315795 85.06885 \n",
       "L 66.689712 84.745029 \n",
       "L 67.063081 84.421682 \n",
       "L 67.435905 84.098807 \n",
       "L 67.808186 83.776401 \n",
       "L 68.179923 83.454467 \n",
       "L 68.551119 83.133002 \n",
       "L 68.921776 82.812004 \n",
       "L 69.291893 82.491474 \n",
       "L 69.661471 82.171409 \n",
       "L 70.030513 81.851809 \n",
       "L 70.39902 81.532673 \n",
       "L 70.766992 81.214 \n",
       "L 71.134431 80.895789 \n",
       "L 71.501337 80.578039 \n",
       "L 71.867713 80.260748 \n",
       "L 72.233559 79.943916 \n",
       "L 72.598875 79.627542 \n",
       "L 72.963665 79.311625 \n",
       "L 73.327928 78.996164 \n",
       "L 73.691666 78.681158 \n",
       "L 74.05488 78.366605 \n",
       "L 74.417571 78.052506 \n",
       "L 74.77974 77.738858 \n",
       "L 75.141388 77.425662 \n",
       "L 75.502517 77.112915 \n",
       "L 75.863127 76.800617 \n",
       "L 76.22322 76.488768 \n",
       "L 76.582796 76.177366 \n",
       "L 76.941857 75.866409 \n",
       "L 77.300405 75.555898 \n",
       "L 77.658439 75.245831 \n",
       "L 78.015962 74.936208 \n",
       "L 78.372974 74.627026 \n",
       "L 78.729476 74.318286 \n",
       "L 79.08547 74.009987 \n",
       "L 79.440956 73.702127 \n",
       "L 79.795935 73.394705 \n",
       "L 80.15041 73.087721 \n",
       "L 80.50438 72.781174 \n",
       "L 80.857847 72.475063 \n",
       "L 81.210812 72.169386 \n",
       "L 81.563276 71.864144 \n",
       "L 81.91524 71.559334 \n",
       "L 82.266705 71.254956 \n",
       "L 82.617672 70.95101 \n",
       "L 82.968142 70.647494 \n",
       "L 83.318117 70.344406 \n",
       "L 83.667597 70.041748 \n",
       "L 84.016584 69.739517 \n",
       "L 84.365077 69.437713 \n",
       "L 84.71308 69.136334 \n",
       "L 85.060592 68.835379 \n",
       "L 85.407614 68.534849 \n",
       "L 85.754148 68.234742 \n",
       "L 86.100195 67.935057 \n",
       "L 86.445755 67.635793 \n",
       "L 86.79083 67.336949 \n",
       "L 87.13542 67.038525 \n",
       "L 87.479527 66.74052 \n",
       "L 87.823153 66.442931 \n",
       "L 88.166296 66.145761 \n",
       "L 88.50896 65.849006 \n",
       "L 88.851145 65.552665 \n",
       "L 89.19285 65.256739 \n",
       "L 89.534079 64.961227 \n",
       "L 89.874832 64.666126 \n",
       "L 90.215109 64.371437 \n",
       "L 90.554912 64.077159 \n",
       "L 90.894242 63.783291 \n",
       "L 91.233099 63.489832 \n",
       "L 91.571485 63.196781 \n",
       "L 91.909402 62.904137 \n",
       "L 92.246848 62.6119 \n",
       "L 92.583826 62.320068 \n",
       "L 92.920337 62.028641 \n",
       "L 93.256381 61.737618 \n",
       "L 93.59196 61.446998 \n",
       "L 93.927074 61.156781 \n",
       "L 94.261726 60.866964 \n",
       "L 94.595914 60.577549 \n",
       "L 94.92964 60.288533 \n",
       "L 95.262907 59.999915 \n",
       "L 95.595714 59.711697 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 66.229204 92.4252 \n",
       "L 66.609324 92.064894 \n",
       "L 66.988862 91.701386 \n",
       "L 67.367817 91.334258 \n",
       "L 67.746191 90.963064 \n",
       "L 68.123987 90.587332 \n",
       "L 68.501205 90.206573 \n",
       "L 68.877848 89.820275 \n",
       "L 69.253919 89.42791 \n",
       "L 69.629422 89.028941 \n",
       "L 70.004359 88.62283 \n",
       "L 70.378735 88.209039 \n",
       "L 70.752557 87.787038 \n",
       "L 71.125828 87.356321 \n",
       "L 71.498557 86.916405 \n",
       "L 71.87075 86.466848 \n",
       "L 72.242415 86.007257 \n",
       "L 72.613561 85.537297 \n",
       "L 72.984198 85.056704 \n",
       "L 73.354334 84.565302 \n",
       "L 73.723981 84.063004 \n",
       "L 74.09315 83.549828 \n",
       "L 74.46185 83.025916 \n",
       "L 74.830094 82.491527 \n",
       "L 75.197894 81.947059 \n",
       "L 75.565259 81.393051 \n",
       "L 75.932201 80.83019 \n",
       "L 76.29873 80.259311 \n",
       "L 76.664854 79.681403 \n",
       "L 77.030583 79.097605 \n",
       "L 77.395924 78.509203 \n",
       "L 77.760881 77.917623 \n",
       "L 78.12546 77.324423 \n",
       "L 78.489662 76.73128 \n",
       "L 78.853488 76.139976 \n",
       "L 79.216937 75.552378 \n",
       "L 79.580004 74.970424 \n",
       "L 79.942684 74.396092 \n",
       "L 80.304968 73.831385 \n",
       "L 80.666845 73.278302 \n",
       "L 81.028304 72.738808 \n",
       "L 81.389329 72.214813 \n",
       "L 81.749904 71.70814 \n",
       "L 82.110012 71.220501 \n",
       "L 82.469632 70.753471 \n",
       "L 82.828744 70.308463 \n",
       "L 83.187327 69.886707 \n",
       "L 83.545358 69.48923 \n",
       "L 83.902816 69.116838 \n",
       "L 84.259679 68.770108 \n",
       "L 84.615927 68.449371 \n",
       "L 84.971537 68.154715 \n",
       "L 85.326492 67.885978 \n",
       "L 85.680775 67.64275 \n",
       "L 86.034369 67.424382 \n",
       "L 86.387262 67.230001 \n",
       "L 86.739443 67.058514 \n",
       "L 87.090904 66.908631 \n",
       "L 87.44164 66.778887 \n",
       "L 87.791647 66.667663 \n",
       "L 88.140925 66.573207 \n",
       "L 88.489478 66.493668 \n",
       "L 88.83731 66.427111 \n",
       "L 89.184429 66.371559 \n",
       "L 89.530846 66.325007 \n",
       "L 89.876572 66.285456 \n",
       "L 90.221622 66.250934 \n",
       "L 90.566011 66.219521 \n",
       "L 90.909757 66.189371 \n",
       "L 91.252877 66.158732 \n",
       "L 91.595391 66.125958 \n",
       "L 91.937318 66.089528 \n",
       "L 92.278676 66.048055 \n",
       "L 92.619486 66.000294 \n",
       "L 92.959766 65.945149 \n",
       "L 93.299534 65.881675 \n",
       "L 93.638809 65.80908 \n",
       "L 93.977606 65.726718 \n",
       "L 94.31594 65.634095 \n",
       "L 94.653826 65.530849 \n",
       "L 94.991277 65.416755 \n",
       "L 95.328302 65.291712 \n",
       "L 95.664914 65.155727 \n",
       "L 96.00112 65.008914 \n",
       "L 96.336926 64.851477 \n",
       "L 96.672339 64.683698 \n",
       "L 97.007363 64.505929 \n",
       "L 97.342001 64.318577 \n",
       "L 97.676256 64.122096 \n",
       "L 98.010128 63.916972 \n",
       "L 98.343617 63.703721 \n",
       "L 98.676723 63.482872 \n",
       "L 99.009445 63.254961 \n",
       "L 99.341779 63.020527 \n",
       "L 99.673724 62.780103 \n",
       "L 100.005278 62.534207 \n",
       "L 100.336438 62.283344 \n",
       "L 100.667199 62.027999 \n",
       "L 100.997559 61.768633 \n",
       "L 101.327515 61.505679 \n",
       "L 101.657063 61.239549 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 72.640761 94.332603 \n",
       "L 73.017845 93.963274 \n",
       "L 73.394368 93.589856 \n",
       "L 73.770331 93.211831 \n",
       "L 74.14574 92.828647 \n",
       "L 74.520599 92.439718 \n",
       "L 74.894912 92.044435 \n",
       "L 75.268686 91.642163 \n",
       "L 75.641925 91.232248 \n",
       "L 76.01464 90.814023 \n",
       "L 76.386834 90.386819 \n",
       "L 76.758519 89.949966 \n",
       "L 77.129704 89.502807 \n",
       "L 77.500396 89.044714 \n",
       "L 77.870609 88.575085 \n",
       "L 78.240353 88.093369 \n",
       "L 78.60964 87.599078 \n",
       "L 78.978482 87.091795 \n",
       "L 79.346893 86.571189 \n",
       "L 79.714884 86.03704 \n",
       "L 80.082469 85.489236 \n",
       "L 80.449663 84.9278 \n",
       "L 80.816475 84.352902 \n",
       "L 81.182918 83.764864 \n",
       "L 81.549005 83.164175 \n",
       "L 81.914744 82.551505 \n",
       "L 82.280143 81.9277 \n",
       "L 82.645212 81.293798 \n",
       "L 83.009955 80.651023 \n",
       "L 83.374373 80.000792 \n",
       "L 83.73847 79.344697 \n",
       "L 84.102241 78.684512 \n",
       "L 84.465684 78.022172 \n",
       "L 84.828789 77.359761 \n",
       "L 85.191547 76.699496 \n",
       "L 85.553944 76.043698 \n",
       "L 85.915962 75.394781 \n",
       "L 86.277584 74.755206 \n",
       "L 86.638785 74.127468 \n",
       "L 86.99954 73.514057 \n",
       "L 87.359822 72.917422 \n",
       "L 87.719601 72.339943 \n",
       "L 88.078846 71.783894 \n",
       "L 88.437523 71.251408 \n",
       "L 88.795598 70.744454 \n",
       "L 89.153038 70.264793 \n",
       "L 89.509808 69.813958 \n",
       "L 89.865874 69.393231 \n",
       "L 90.221205 69.003622 \n",
       "L 90.57577 68.645847 \n",
       "L 90.92954 68.320321 \n",
       "L 91.28249 68.027156 \n",
       "L 91.634598 67.766145 \n",
       "L 91.985843 67.536781 \n",
       "L 92.336212 67.338254 \n",
       "L 92.685693 67.169469 \n",
       "L 93.034279 67.029066 \n",
       "L 93.381968 66.915437 \n",
       "L 93.728762 66.826753 \n",
       "L 94.074667 66.760994 \n",
       "L 94.419693 66.715976 \n",
       "L 94.763855 66.68939 \n",
       "L 95.107171 66.67883 \n",
       "L 95.449662 66.681827 \n",
       "L 95.791354 66.695887 \n",
       "L 96.132272 66.71852 \n",
       "L 96.472447 66.747274 \n",
       "L 96.811908 66.779759 \n",
       "L 97.150688 66.813679 \n",
       "L 97.48882 66.846856 \n",
       "L 97.826336 66.877246 \n",
       "L 98.16327 66.902958 \n",
       "L 98.499653 66.922272 \n",
       "L 98.835516 66.933643 \n",
       "L 99.170889 66.935711 \n",
       "L 99.5058 66.927307 \n",
       "L 99.840273 66.907448 \n",
       "L 100.174336 66.875337 \n",
       "L 100.508006 66.830361 \n",
       "L 100.841305 66.772077 \n",
       "L 101.17425 66.700207 \n",
       "L 101.506852 66.614627 \n",
       "L 101.839126 66.515353 \n",
       "L 102.171081 66.402526 \n",
       "L 102.502722 66.276403 \n",
       "L 102.834055 66.137337 \n",
       "L 103.165085 65.985765 \n",
       "L 103.49581 65.822197 \n",
       "L 103.826231 65.647196 \n",
       "L 104.156347 65.461368 \n",
       "L 104.486153 65.265355 \n",
       "L 104.815646 65.059813 \n",
       "L 105.144821 64.84541 \n",
       "L 105.47367 64.622816 \n",
       "L 105.802189 64.392692 \n",
       "L 106.130369 64.155683 \n",
       "L 106.458206 63.912415 \n",
       "L 106.78569 63.663491 \n",
       "L 107.112816 63.409483 \n",
       "L 107.439576 63.15093 \n",
       "L 107.765963 62.888343 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 79.108182 96.352444 \n",
       "L 79.482183 95.990413 \n",
       "L 79.855641 95.625788 \n",
       "L 80.228557 95.258221 \n",
       "L 80.600939 94.887337 \n",
       "L 80.972792 94.51274 \n",
       "L 81.34412 94.13402 \n",
       "L 81.71493 93.750746 \n",
       "L 82.08523 93.362478 \n",
       "L 82.455027 92.968764 \n",
       "L 82.824328 92.569153 \n",
       "L 83.193141 92.163194 \n",
       "L 83.561478 91.750443 \n",
       "L 83.929343 91.330475 \n",
       "L 84.29675 90.902888 \n",
       "L 84.663707 90.467309 \n",
       "L 85.030224 90.02341 \n",
       "L 85.39631 89.570912 \n",
       "L 85.761978 89.109593 \n",
       "L 86.127235 88.639307 \n",
       "L 86.492091 88.159981 \n",
       "L 86.856557 87.671633 \n",
       "L 87.220637 87.174382 \n",
       "L 87.584342 86.668446 \n",
       "L 87.947677 86.15416 \n",
       "L 88.310646 85.631978 \n",
       "L 88.673252 85.102477 \n",
       "L 89.035499 84.566356 \n",
       "L 89.397384 84.024446 \n",
       "L 89.758905 83.477704 \n",
       "L 90.120059 82.927205 \n",
       "L 90.480838 82.374148 \n",
       "L 90.841232 81.819836 \n",
       "L 91.20123 81.265675 \n",
       "L 91.560818 80.713158 \n",
       "L 91.919979 80.163847 \n",
       "L 92.278694 79.619366 \n",
       "L 92.636943 79.08137 \n",
       "L 92.994702 78.551533 \n",
       "L 93.351946 78.03153 \n",
       "L 93.708649 77.523004 \n",
       "L 94.064783 77.02755 \n",
       "L 94.420321 76.546695 \n",
       "L 94.775233 76.081868 \n",
       "L 95.129491 75.634387 \n",
       "L 95.483067 75.205431 \n",
       "L 95.835932 74.796029 \n",
       "L 96.188062 74.407039 \n",
       "L 96.539432 74.039133 \n",
       "L 96.890018 73.692794 \n",
       "L 97.239803 73.368298 \n",
       "L 97.588767 73.065718 \n",
       "L 97.936898 72.784918 \n",
       "L 98.284184 72.525557 \n",
       "L 98.630619 72.287092 \n",
       "L 98.9762 72.068791 \n",
       "L 99.320927 71.869742 \n",
       "L 99.664803 71.688867 \n",
       "L 100.007837 71.524942 \n",
       "L 100.35004 71.376611 \n",
       "L 100.691426 71.242411 \n",
       "L 101.032013 71.120791 \n",
       "L 101.371821 71.010137 \n",
       "L 101.710874 70.908792 \n",
       "L 102.049196 70.815078 \n",
       "L 102.386813 70.727322 \n",
       "L 102.723755 70.643874 \n",
       "L 103.060049 70.563127 \n",
       "L 103.395725 70.483533 \n",
       "L 103.730813 70.403623 \n",
       "L 104.065341 70.322021 \n",
       "L 104.399339 70.237449 \n",
       "L 104.732832 70.148745 \n",
       "L 105.065849 70.054865 \n",
       "L 105.398414 69.954889 \n",
       "L 105.730548 69.848024 \n",
       "L 106.062274 69.733604 \n",
       "L 106.393611 69.611087 \n",
       "L 106.724575 69.480055 \n",
       "L 107.05518 69.340206 \n",
       "L 107.385441 69.191349 \n",
       "L 107.715365 69.033397 \n",
       "L 108.044962 68.866356 \n",
       "L 108.374239 68.690319 \n",
       "L 108.703199 68.505456 \n",
       "L 109.031844 68.312004 \n",
       "L 109.360176 68.110254 \n",
       "L 109.688194 67.90055 \n",
       "L 110.015896 67.683269 \n",
       "L 110.34328 67.458819 \n",
       "L 110.67034 67.227633 \n",
       "L 110.997072 66.990152 \n",
       "L 111.323472 66.746826 \n",
       "L 111.649533 66.498108 \n",
       "L 111.975249 66.244441 \n",
       "L 112.300613 65.986263 \n",
       "L 112.62562 65.723995 \n",
       "L 112.950262 65.458043 \n",
       "L 113.274533 65.188795 \n",
       "L 113.598429 64.916613 \n",
       "L 113.921941 64.641843 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 85.631215 98.399829 \n",
       "L 86.001946 98.046849 \n",
       "L 86.372134 97.693033 \n",
       "L 86.741781 97.338229 \n",
       "L 87.11089 96.982271 \n",
       "L 87.479468 96.624986 \n",
       "L 87.847515 96.266195 \n",
       "L 88.215038 95.905711 \n",
       "L 88.582041 95.54334 \n",
       "L 88.94853 95.178884 \n",
       "L 89.314508 94.812149 \n",
       "L 89.679981 94.442936 \n",
       "L 90.044957 94.071051 \n",
       "L 90.409437 93.69631 \n",
       "L 90.77343 93.318537 \n",
       "L 91.136941 92.937569 \n",
       "L 91.499975 92.553266 \n",
       "L 91.862539 92.165505 \n",
       "L 92.224637 91.774189 \n",
       "L 92.586275 91.379256 \n",
       "L 92.947457 90.980677 \n",
       "L 93.30819 90.578458 \n",
       "L 93.668474 90.172653 \n",
       "L 94.028315 89.763359 \n",
       "L 94.387715 89.350721 \n",
       "L 94.746675 88.93494 \n",
       "L 95.105197 88.516267 \n",
       "L 95.463279 88.095007 \n",
       "L 95.820921 87.671524 \n",
       "L 96.17812 87.246235 \n",
       "L 96.534873 86.819607 \n",
       "L 96.891173 86.392163 \n",
       "L 97.247015 85.964468 \n",
       "L 97.602393 85.537135 \n",
       "L 97.957295 85.110811 \n",
       "L 98.311713 84.686174 \n",
       "L 98.665636 84.263928 \n",
       "L 99.019051 83.84479 \n",
       "L 99.371946 83.429486 \n",
       "L 99.724306 83.018738 \n",
       "L 100.076118 82.613259 \n",
       "L 100.427367 82.213737 \n",
       "L 100.778038 81.820832 \n",
       "L 101.128117 81.435161 \n",
       "L 101.477588 81.057293 \n",
       "L 101.826437 80.687736 \n",
       "L 102.174652 80.326934 \n",
       "L 102.52222 79.975255 \n",
       "L 102.869129 79.632991 \n",
       "L 103.215369 79.300346 \n",
       "L 103.560933 78.977442 \n",
       "L 103.905813 78.664309 \n",
       "L 104.250004 78.360887 \n",
       "L 104.593502 78.067028 \n",
       "L 104.936308 77.782498 \n",
       "L 105.278421 77.506982 \n",
       "L 105.619844 77.240084 \n",
       "L 105.960582 76.981338 \n",
       "L 106.300642 76.730217 \n",
       "L 106.640031 76.486132 \n",
       "L 106.97876 76.248452 \n",
       "L 107.316841 76.016505 \n",
       "L 107.654286 75.789593 \n",
       "L 107.99111 75.566997 \n",
       "L 108.327328 75.347991 \n",
       "L 108.662953 75.131847 \n",
       "L 108.998005 74.917852 \n",
       "L 109.332498 74.705305 \n",
       "L 109.66645 74.493534 \n",
       "L 109.999877 74.281901 \n",
       "L 110.332795 74.069805 \n",
       "L 110.66522 73.85669 \n",
       "L 110.997166 73.642049 \n",
       "L 111.328649 73.425425 \n",
       "L 111.659681 73.206417 \n",
       "L 111.990274 72.984677 \n",
       "L 112.320439 72.759913 \n",
       "L 112.650188 72.531887 \n",
       "L 112.979528 72.300414 \n",
       "L 113.308466 72.065361 \n",
       "L 113.637011 71.826641 \n",
       "L 113.965166 71.584216 \n",
       "L 114.292935 71.338085 \n",
       "L 114.620323 71.088287 \n",
       "L 114.947329 70.834895 \n",
       "L 115.273957 70.578009 \n",
       "L 115.600207 70.317755 \n",
       "L 115.926077 70.054282 \n",
       "L 116.251566 69.787753 \n",
       "L 116.576674 69.518343 \n",
       "L 116.901398 69.246241 \n",
       "L 117.225734 68.971639 \n",
       "L 117.549683 68.69473 \n",
       "L 117.873238 68.415713 \n",
       "L 118.196399 68.134779 \n",
       "L 118.51916 67.852119 \n",
       "L 118.841521 67.567914 \n",
       "L 119.163476 67.282343 \n",
       "L 119.485023 66.995574 \n",
       "L 119.806161 66.707762 \n",
       "L 120.126883 66.419061 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 92.210887 100.43245 \n",
       "L 92.578224 100.083001 \n",
       "L 92.945013 99.733674 \n",
       "L 93.311254 99.384423 \n",
       "L 93.67695 99.035198 \n",
       "L 94.042105 98.685946 \n",
       "L 94.406719 98.336616 \n",
       "L 94.770795 97.987149 \n",
       "L 95.134336 97.637487 \n",
       "L 95.497346 97.28757 \n",
       "L 95.859825 96.937342 \n",
       "L 96.221777 96.58674 \n",
       "L 96.583206 96.235708 \n",
       "L 96.944112 95.884189 \n",
       "L 97.3045 95.532132 \n",
       "L 97.664373 95.179486 \n",
       "L 98.023731 94.826211 \n",
       "L 98.382579 94.472267 \n",
       "L 98.74092 94.117627 \n",
       "L 99.098755 93.762271 \n",
       "L 99.456086 93.40619 \n",
       "L 99.812918 93.049385 \n",
       "L 100.169249 92.691872 \n",
       "L 100.525083 92.33368 \n",
       "L 100.880422 91.974849 \n",
       "L 101.235265 91.61544 \n",
       "L 101.589613 91.255527 \n",
       "L 101.943468 90.8952 \n",
       "L 102.296828 90.534565 \n",
       "L 102.649692 90.173747 \n",
       "L 103.002062 89.812883 \n",
       "L 103.353933 89.452128 \n",
       "L 103.705305 89.091647 \n",
       "L 104.056176 88.731623 \n",
       "L 104.406542 88.372247 \n",
       "L 104.7564 88.013717 \n",
       "L 105.105747 87.656243 \n",
       "L 105.454579 87.300035 \n",
       "L 105.802893 86.945308 \n",
       "L 106.150682 86.592274 \n",
       "L 106.497944 86.241144 \n",
       "L 106.844673 85.89212 \n",
       "L 107.190864 85.545396 \n",
       "L 107.536515 85.201155 \n",
       "L 107.881619 84.859562 \n",
       "L 108.226174 84.520768 \n",
       "L 108.570174 84.184903 \n",
       "L 108.913618 83.852075 \n",
       "L 109.2565 83.522369 \n",
       "L 109.598821 83.195845 \n",
       "L 109.940577 82.872538 \n",
       "L 110.281767 82.552456 \n",
       "L 110.62239 82.23558 \n",
       "L 110.962447 81.921868 \n",
       "L 111.301938 81.611247 \n",
       "L 111.640865 81.303625 \n",
       "L 111.97923 80.998885 \n",
       "L 112.317036 80.696888 \n",
       "L 112.654286 80.397476 \n",
       "L 112.990985 80.100477 \n",
       "L 113.327136 79.805702 \n",
       "L 113.662746 79.512954 \n",
       "L 113.99782 79.222022 \n",
       "L 114.332363 78.932696 \n",
       "L 114.666383 78.644759 \n",
       "L 114.999884 78.357997 \n",
       "L 115.332876 78.072194 \n",
       "L 115.665362 77.787146 \n",
       "L 115.997351 77.502651 \n",
       "L 116.328849 77.218519 \n",
       "L 116.659862 76.934572 \n",
       "L 116.990397 76.650645 \n",
       "L 117.320459 76.366585 \n",
       "L 117.650054 76.082256 \n",
       "L 117.979188 75.797539 \n",
       "L 118.307864 75.51233 \n",
       "L 118.636087 75.226541 \n",
       "L 118.963864 74.9401 \n",
       "L 119.291194 74.652952 \n",
       "L 119.618082 74.365055 \n",
       "L 119.944532 74.076384 \n",
       "L 120.270545 73.786926 \n",
       "L 120.596123 73.49668 \n",
       "L 120.921268 73.205656 \n",
       "L 121.245981 72.913876 \n",
       "L 121.570262 72.621369 \n",
       "L 121.894113 72.328169 \n",
       "L 122.217534 72.034322 \n",
       "L 122.540524 71.739875 \n",
       "L 122.863085 71.444879 \n",
       "L 123.185214 71.149389 \n",
       "L 123.506913 70.853462 \n",
       "L 123.828181 70.557153 \n",
       "L 124.149017 70.260522 \n",
       "L 124.469421 69.963626 \n",
       "L 124.789391 69.666518 \n",
       "L 125.10893 69.369253 \n",
       "L 125.428033 69.071884 \n",
       "L 125.746703 68.774459 \n",
       "L 126.064938 68.477023 \n",
       "L 126.382738 68.179622 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 28.892801 81.271898 \n",
       "L 29.499809 81.458195 \n",
       "L 30.107335 81.644918 \n",
       "L 30.715376 81.832086 \n",
       "L 31.323936 82.019716 \n",
       "L 31.933018 82.207826 \n",
       "L 32.542617 82.396429 \n",
       "L 33.152737 82.585538 \n",
       "L 33.763378 82.775162 \n",
       "L 34.374542 82.965308 \n",
       "L 34.986226 83.155976 \n",
       "L 35.598431 83.347163 \n",
       "L 36.21116 83.538861 \n",
       "L 36.824407 83.731054 \n",
       "L 37.438175 83.923722 \n",
       "L 38.052465 84.116835 \n",
       "L 38.667272 84.310358 \n",
       "L 39.282597 84.504248 \n",
       "L 39.89844 84.698454 \n",
       "L 40.514797 84.892914 \n",
       "L 41.131668 85.087564 \n",
       "L 41.749052 85.282329 \n",
       "L 42.366946 85.477126 \n",
       "L 42.985349 85.671869 \n",
       "L 43.604262 85.866464 \n",
       "L 44.223679 86.060814 \n",
       "L 44.8436 86.254818 \n",
       "L 45.464027 86.448373 \n",
       "L 46.084955 86.641379 \n",
       "L 46.706384 86.833733 \n",
       "L 47.328316 87.025338 \n",
       "L 47.950747 87.216101 \n",
       "L 48.57368 87.405938 \n",
       "L 49.197115 87.594771 \n",
       "L 49.821051 87.782536 \n",
       "L 50.445492 87.969178 \n",
       "L 51.070438 88.154659 \n",
       "L 51.695892 88.338955 \n",
       "L 52.321856 88.522059 \n",
       "L 52.948332 88.703979 \n",
       "L 53.575324 88.884744 \n",
       "L 54.202836 89.0644 \n",
       "L 54.830871 89.243012 \n",
       "L 55.459434 89.420659 \n",
       "L 56.088527 89.597441 \n",
       "L 56.718156 89.773471 \n",
       "L 57.348325 89.948876 \n",
       "L 57.979037 90.123798 \n",
       "L 58.610297 90.298386 \n",
       "L 59.242108 90.472797 \n",
       "L 59.874475 90.647197 \n",
       "L 60.507399 90.82175 \n",
       "L 61.140885 90.996624 \n",
       "L 61.774934 91.171982 \n",
       "L 62.40955 91.347984 \n",
       "L 63.044733 91.524782 \n",
       "L 63.680486 91.702516 \n",
       "L 64.316809 91.881317 \n",
       "L 64.953703 92.0613 \n",
       "L 65.591168 92.242567 \n",
       "L 66.229204 92.4252 \n",
       "L 66.867811 92.609267 \n",
       "L 67.506987 92.794815 \n",
       "L 68.146732 92.981876 \n",
       "L 68.787046 93.170462 \n",
       "L 69.427924 93.360566 \n",
       "L 70.069368 93.552169 \n",
       "L 70.711375 93.745231 \n",
       "L 71.353944 93.939703 \n",
       "L 71.997073 94.135518 \n",
       "L 72.640761 94.332603 \n",
       "L 73.285007 94.530872 \n",
       "L 73.929808 94.730233 \n",
       "L 74.575166 94.930589 \n",
       "L 75.221079 95.13184 \n",
       "L 75.867544 95.333881 \n",
       "L 76.514564 95.536612 \n",
       "L 77.162139 95.739929 \n",
       "L 77.810265 95.943736 \n",
       "L 78.458946 96.147937 \n",
       "L 79.108182 96.352444 \n",
       "L 79.757972 96.557174 \n",
       "L 80.408317 96.762052 \n",
       "L 81.059221 96.967009 \n",
       "L 81.71068 97.171985 \n",
       "L 82.362698 97.376927 \n",
       "L 83.015278 97.58179 \n",
       "L 83.668417 97.786538 \n",
       "L 84.322118 97.99114 \n",
       "L 84.976385 98.195576 \n",
       "L 85.631215 98.399829 \n",
       "L 86.286612 98.603888 \n",
       "L 86.942578 98.807752 \n",
       "L 87.599112 99.01142 \n",
       "L 88.256216 99.214897 \n",
       "L 88.913891 99.418193 \n",
       "L 89.572141 99.62132 \n",
       "L 90.230964 99.824292 \n",
       "L 90.890361 100.027125 \n",
       "L 91.550336 100.229839 \n",
       "L 92.210887 100.43245 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 32.84695 78.160058 \n",
       "L 33.451458 78.351179 \n",
       "L 34.056501 78.543592 \n",
       "L 34.662076 78.737374 \n",
       "L 35.268187 78.932601 \n",
       "L 35.874836 79.129342 \n",
       "L 36.482021 79.327657 \n",
       "L 37.089744 79.527597 \n",
       "L 37.698005 79.7292 \n",
       "L 38.306804 79.932491 \n",
       "L 38.916137 80.137473 \n",
       "L 39.526003 80.344132 \n",
       "L 40.136401 80.552432 \n",
       "L 40.747325 80.762307 \n",
       "L 41.358773 80.973669 \n",
       "L 41.970741 81.186398 \n",
       "L 42.58322 81.400342 \n",
       "L 43.196207 81.615317 \n",
       "L 43.809697 81.831108 \n",
       "L 44.423679 82.047465 \n",
       "L 45.038148 82.264105 \n",
       "L 45.653098 82.480717 \n",
       "L 46.268518 82.696957 \n",
       "L 46.884402 82.912456 \n",
       "L 47.500744 83.126825 \n",
       "L 48.117534 83.339651 \n",
       "L 48.734766 83.550514 \n",
       "L 49.352436 83.758984 \n",
       "L 49.970537 83.964629 \n",
       "L 50.589064 84.167026 \n",
       "L 51.208015 84.365765 \n",
       "L 51.827388 84.560458 \n",
       "L 52.447184 84.750747 \n",
       "L 53.067401 84.936309 \n",
       "L 53.688042 85.116869 \n",
       "L 54.309113 85.292202 \n",
       "L 54.930617 85.462144 \n",
       "L 55.552562 85.626591 \n",
       "L 56.174957 85.785513 \n",
       "L 56.79781 85.93895 \n",
       "L 57.421132 86.087016 \n",
       "L 58.044936 86.229904 \n",
       "L 58.669233 86.367879 \n",
       "L 59.294038 86.501283 \n",
       "L 59.919363 86.630526 \n",
       "L 60.545223 86.756083 \n",
       "L 61.17163 86.87849 \n",
       "L 61.798598 86.998333 \n",
       "L 62.426139 87.116241 \n",
       "L 63.054265 87.232877 \n",
       "L 63.682989 87.348927 \n",
       "L 64.312317 87.465088 \n",
       "L 64.942259 87.582061 \n",
       "L 65.572821 87.700533 \n",
       "L 66.204008 87.821173 \n",
       "L 66.835825 87.944618 \n",
       "L 67.468272 88.071462 \n",
       "L 68.101351 88.202251 \n",
       "L 68.735061 88.337472 \n",
       "L 69.369398 88.477547 \n",
       "L 70.004359 88.62283 \n",
       "L 70.639938 88.773601 \n",
       "L 71.276131 88.930063 \n",
       "L 71.91293 89.092343 \n",
       "L 72.550327 89.260494 \n",
       "L 73.188314 89.434492 \n",
       "L 73.826885 89.614248 \n",
       "L 74.46603 89.799602 \n",
       "L 75.105742 89.990337 \n",
       "L 75.746013 90.186182 \n",
       "L 76.386834 90.386819 \n",
       "L 77.028202 90.591892 \n",
       "L 77.670108 90.801014 \n",
       "L 78.312549 91.013774 \n",
       "L 78.95552 91.229748 \n",
       "L 79.599016 91.448501 \n",
       "L 80.243035 91.669601 \n",
       "L 80.887578 91.892622 \n",
       "L 81.53264 92.117147 \n",
       "L 82.178223 92.342782 \n",
       "L 82.824328 92.569153 \n",
       "L 83.470954 92.79591 \n",
       "L 84.118104 93.022735 \n",
       "L 84.765782 93.249341 \n",
       "L 85.413987 93.475472 \n",
       "L 86.062724 93.700906 \n",
       "L 86.711998 93.925456 \n",
       "L 87.361808 94.148964 \n",
       "L 88.012161 94.371307 \n",
       "L 88.663061 94.592391 \n",
       "L 89.314508 94.812149 \n",
       "L 89.966507 95.03054 \n",
       "L 90.619064 95.247547 \n",
       "L 91.272177 95.463173 \n",
       "L 91.925852 95.67744 \n",
       "L 92.580091 95.890385 \n",
       "L 93.234898 96.102058 \n",
       "L 93.890271 96.312518 \n",
       "L 94.546216 96.521835 \n",
       "L 95.202735 96.730083 \n",
       "L 95.859825 96.937342 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 36.747008 75.178391 \n",
       "L 37.349346 75.387006 \n",
       "L 37.952264 75.599247 \n",
       "L 38.555761 75.815351 \n",
       "L 39.159841 76.035545 \n",
       "L 39.764508 76.260039 \n",
       "L 40.369757 76.489016 \n",
       "L 40.97559 76.72263 \n",
       "L 41.582004 76.960998 \n",
       "L 42.188997 77.20419 \n",
       "L 42.79656 77.452219 \n",
       "L 43.404688 77.70504 \n",
       "L 44.013373 77.962537 \n",
       "L 44.6226 78.224515 \n",
       "L 45.23236 78.490698 \n",
       "L 45.842639 78.760719 \n",
       "L 46.453417 79.034116 \n",
       "L 47.064679 79.310329 \n",
       "L 47.676408 79.588699 \n",
       "L 48.28858 79.868463 \n",
       "L 48.901176 80.148763 \n",
       "L 49.514176 80.428645 \n",
       "L 50.127556 80.707065 \n",
       "L 50.741295 80.982902 \n",
       "L 51.355375 81.254964 \n",
       "L 51.969774 81.522006 \n",
       "L 52.584474 81.782742 \n",
       "L 53.199461 82.035866 \n",
       "L 53.814721 82.280068 \n",
       "L 54.430242 82.514061 \n",
       "L 55.046019 82.736598 \n",
       "L 55.662047 82.946499 \n",
       "L 56.278326 83.142674 \n",
       "L 56.894861 83.324144 \n",
       "L 57.511659 83.49007 \n",
       "L 58.128734 83.639767 \n",
       "L 58.746101 83.772729 \n",
       "L 59.363781 83.888646 \n",
       "L 59.981799 83.987414 \n",
       "L 60.60018 84.069152 \n",
       "L 61.218955 84.134205 \n",
       "L 61.838157 84.183147 \n",
       "L 62.45782 84.216786 \n",
       "L 63.077979 84.236149 \n",
       "L 63.698671 84.242478 \n",
       "L 64.31993 84.237217 \n",
       "L 64.941793 84.221988 \n",
       "L 65.564292 84.19857 \n",
       "L 66.187458 84.168878 \n",
       "L 66.81132 84.134928 \n",
       "L 67.435905 84.098807 \n",
       "L 68.061231 84.062643 \n",
       "L 68.687316 84.028568 \n",
       "L 69.314175 83.998688 \n",
       "L 69.941815 83.975043 \n",
       "L 70.570241 83.95958 \n",
       "L 71.199451 83.954123 \n",
       "L 71.829441 83.960342 \n",
       "L 72.460202 83.979732 \n",
       "L 73.091721 84.013591 \n",
       "L 73.723981 84.063004 \n",
       "L 74.356964 84.128831 \n",
       "L 74.990648 84.211703 \n",
       "L 75.625011 84.312013 \n",
       "L 76.260027 84.429924 \n",
       "L 76.895671 84.565373 \n",
       "L 77.53192 84.71808 \n",
       "L 78.168746 84.887564 \n",
       "L 78.806128 85.073162 \n",
       "L 79.444043 85.274043 \n",
       "L 80.082469 85.489236 \n",
       "L 80.72139 85.717649 \n",
       "L 81.360788 85.958096 \n",
       "L 82.000651 86.20932 \n",
       "L 82.640967 86.47002 \n",
       "L 83.281727 86.738867 \n",
       "L 83.922926 87.014537 \n",
       "L 84.564563 87.295723 \n",
       "L 85.206634 87.581157 \n",
       "L 85.849141 87.869624 \n",
       "L 86.492091 88.159981 \n",
       "L 87.135484 88.451163 \n",
       "L 87.77933 88.742196 \n",
       "L 88.423637 89.032201 \n",
       "L 89.068411 89.320397 \n",
       "L 89.713664 89.606108 \n",
       "L 90.359407 89.888761 \n",
       "L 91.005646 90.167877 \n",
       "L 91.652394 90.443079 \n",
       "L 92.299663 90.71408 \n",
       "L 92.947457 90.980677 \n",
       "L 93.595789 91.242746 \n",
       "L 94.244668 91.500236 \n",
       "L 94.894097 91.753155 \n",
       "L 95.544087 92.001572 \n",
       "L 96.194644 92.245598 \n",
       "L 96.845774 92.485387 \n",
       "L 97.497478 92.721122 \n",
       "L 98.149763 92.953014 \n",
       "L 98.802632 93.18129 \n",
       "L 99.456086 93.40619 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 40.594116 72.350101 \n",
       "L 41.194594 72.592457 \n",
       "L 41.795721 72.842653 \n",
       "L 42.397498 73.101214 \n",
       "L 42.999929 73.368641 \n",
       "L 43.603017 73.645393 \n",
       "L 44.206757 73.931876 \n",
       "L 44.811147 74.228429 \n",
       "L 45.416179 74.535304 \n",
       "L 46.021846 74.852652 \n",
       "L 46.628132 75.180501 \n",
       "L 47.23502 75.518745 \n",
       "L 47.842494 75.867123 \n",
       "L 48.450525 76.225202 \n",
       "L 49.05909 76.592365 \n",
       "L 49.668159 76.967797 \n",
       "L 50.277696 77.350471 \n",
       "L 50.887666 77.739147 \n",
       "L 51.498033 78.132365 \n",
       "L 52.108754 78.528437 \n",
       "L 52.719789 78.925467 \n",
       "L 53.331099 79.321347 \n",
       "L 53.942637 79.713774 \n",
       "L 54.554367 80.100274 \n",
       "L 55.166251 80.478221 \n",
       "L 55.778249 80.84487 \n",
       "L 56.390333 81.197391 \n",
       "L 57.002476 81.532906 \n",
       "L 57.614655 81.848536 \n",
       "L 58.226854 82.141446 \n",
       "L 58.839066 82.408895 \n",
       "L 59.451287 82.648285 \n",
       "L 60.063526 82.857215 \n",
       "L 60.675797 83.033533 \n",
       "L 61.28812 83.175383 \n",
       "L 61.900528 83.281255 \n",
       "L 62.513059 83.350024 \n",
       "L 63.125758 83.380993 \n",
       "L 63.738678 83.373923 \n",
       "L 64.351876 83.329055 \n",
       "L 64.965416 83.247127 \n",
       "L 65.579363 83.129386 \n",
       "L 66.193788 82.977583 \n",
       "L 66.808761 82.793961 \n",
       "L 67.424351 82.581236 \n",
       "L 68.040628 82.342565 \n",
       "L 68.657656 82.081509 \n",
       "L 69.275498 81.801982 \n",
       "L 69.894209 81.508195 \n",
       "L 70.513839 81.204593 \n",
       "L 71.134431 80.895789 \n",
       "L 71.756014 80.586491 \n",
       "L 72.378616 80.281426 \n",
       "L 73.00225 79.985263 \n",
       "L 73.626923 79.702544 \n",
       "L 74.252629 79.437604 \n",
       "L 74.879357 79.194511 \n",
       "L 75.507084 78.977 \n",
       "L 76.13578 78.788415 \n",
       "L 76.765407 78.63167 \n",
       "L 77.395924 78.509203 \n",
       "L 78.027281 78.422954 \n",
       "L 78.659426 78.374349 \n",
       "L 79.292306 78.364285 \n",
       "L 79.925864 78.393143 \n",
       "L 80.560044 78.460797 \n",
       "L 81.194792 78.566638 \n",
       "L 81.830055 78.709604 \n",
       "L 82.465786 78.888219 \n",
       "L 83.101938 79.100638 \n",
       "L 83.73847 79.344697 \n",
       "L 84.37535 79.617962 \n",
       "L 85.012547 79.917789 \n",
       "L 85.650039 80.241376 \n",
       "L 86.28781 80.585815 \n",
       "L 86.925848 80.948151 \n",
       "L 87.564148 81.325426 \n",
       "L 88.202712 81.714729 \n",
       "L 88.841544 82.113235 \n",
       "L 89.480654 82.518242 \n",
       "L 90.120059 82.927205 \n",
       "L 90.759772 83.337757 \n",
       "L 91.399816 83.747731 \n",
       "L 92.040214 84.155178 \n",
       "L 92.680985 84.558366 \n",
       "L 93.322157 84.955796 \n",
       "L 93.963756 85.346197 \n",
       "L 94.605803 85.728515 \n",
       "L 95.248322 86.101913 \n",
       "L 95.89134 86.465759 \n",
       "L 96.534873 86.819607 \n",
       "L 97.178943 87.163187 \n",
       "L 97.823569 87.496389 \n",
       "L 98.468765 87.819236 \n",
       "L 99.114546 88.13188 \n",
       "L 99.760924 88.434575 \n",
       "L 100.407911 88.727662 \n",
       "L 101.055512 89.011551 \n",
       "L 101.703734 89.286709 \n",
       "L 102.352584 89.553642 \n",
       "L 103.002062 89.812883 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 44.386339 69.591697 \n",
       "L 44.984849 69.871773 \n",
       "L 45.58406 70.164355 \n",
       "L 46.183974 70.470284 \n",
       "L 46.784594 70.79036 \n",
       "L 47.385921 71.12532 \n",
       "L 47.987945 71.475812 \n",
       "L 48.590662 71.842375 \n",
       "L 49.194058 72.225407 \n",
       "L 49.798118 72.625146 \n",
       "L 50.402816 73.041631 \n",
       "L 51.008129 73.474683 \n",
       "L 51.614027 73.923876 \n",
       "L 52.22047 74.388507 \n",
       "L 52.827419 74.867579 \n",
       "L 53.434832 75.359775 \n",
       "L 54.042656 75.86344 \n",
       "L 54.65084 76.376579 \n",
       "L 55.259332 76.896842 \n",
       "L 55.868071 77.421522 \n",
       "L 56.477002 77.947573 \n",
       "L 57.086069 78.471617 \n",
       "L 57.695213 78.989964 \n",
       "L 58.304383 79.498655 \n",
       "L 58.91353 79.993486 \n",
       "L 59.522607 80.470069 \n",
       "L 60.131577 80.923882 \n",
       "L 60.740411 81.350327 \n",
       "L 61.349086 81.744809 \n",
       "L 61.957589 82.102799 \n",
       "L 62.56592 82.419922 \n",
       "L 63.174086 82.692029 \n",
       "L 63.78211 82.915288 \n",
       "L 64.390023 83.086253 \n",
       "L 64.997867 83.201957 \n",
       "L 65.6057 83.259975 \n",
       "L 66.213585 83.258496 \n",
       "L 66.8216 83.196385 \n",
       "L 67.429825 83.07323 \n",
       "L 68.038353 82.889387 \n",
       "L 68.64728 82.646003 \n",
       "L 69.256705 82.345029 \n",
       "L 69.866731 81.989224 \n",
       "L 70.477459 81.582133 \n",
       "L 71.088987 81.12806 \n",
       "L 71.701411 80.632023 \n",
       "L 72.314819 80.099686 \n",
       "L 72.929291 79.537289 \n",
       "L 73.544899 78.951562 \n",
       "L 74.161699 78.34962 \n",
       "L 74.77974 77.738858 \n",
       "L 75.399051 77.126842 \n",
       "L 76.01965 76.521176 \n",
       "L 76.641542 75.929392 \n",
       "L 77.264715 75.358824 \n",
       "L 77.889142 74.81649 \n",
       "L 78.514783 74.308984 \n",
       "L 79.141588 73.842372 \n",
       "L 79.769493 73.422099 \n",
       "L 80.398425 73.052915 \n",
       "L 81.028304 72.738808 \n",
       "L 81.659042 72.482959 \n",
       "L 82.290551 72.287712 \n",
       "L 82.922738 72.15456 \n",
       "L 83.555512 72.084153 \n",
       "L 84.188781 72.076313 \n",
       "L 84.822463 72.130077 \n",
       "L 85.456476 72.24374 \n",
       "L 86.090748 72.414926 \n",
       "L 86.725216 72.640655 \n",
       "L 87.359822 72.917422 \n",
       "L 87.994525 73.241288 \n",
       "L 88.629286 73.607965 \n",
       "L 89.264084 74.012915 \n",
       "L 89.898903 74.451424 \n",
       "L 90.533739 74.918703 \n",
       "L 91.168598 75.409962 \n",
       "L 91.803495 75.920486 \n",
       "L 92.438449 76.445706 \n",
       "L 93.073488 76.981249 \n",
       "L 93.708649 77.523004 \n",
       "L 94.343966 78.067145 \n",
       "L 94.979482 78.610179 \n",
       "L 95.615245 79.148961 \n",
       "L 96.251294 79.680702 \n",
       "L 96.887677 80.20299 \n",
       "L 97.524441 80.713779 \n",
       "L 98.161626 81.21138 \n",
       "L 98.799275 81.694451 \n",
       "L 99.437429 82.161981 \n",
       "L 100.076118 82.613259 \n",
       "L 100.715379 83.047858 \n",
       "L 101.355241 83.465607 \n",
       "L 101.995725 83.866552 \n",
       "L 102.636855 84.250943 \n",
       "L 103.278648 84.619193 \n",
       "L 103.92112 84.971858 \n",
       "L 104.564278 85.309601 \n",
       "L 105.208132 85.633177 \n",
       "L 105.852687 85.943405 \n",
       "L 106.497944 86.241144 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 48.120516 66.725034 \n",
       "L 48.716404 67.020524 \n",
       "L 49.312983 67.330585 \n",
       "L 49.910251 67.6562 \n",
       "L 50.50821 67.998302 \n",
       "L 51.106856 68.357748 \n",
       "L 51.70618 68.735293 \n",
       "L 52.306171 69.131564 \n",
       "L 52.906813 69.547022 \n",
       "L 53.508088 69.981943 \n",
       "L 54.109966 70.436367 \n",
       "L 54.712418 70.910085 \n",
       "L 55.315413 71.402593 \n",
       "L 55.918907 71.913067 \n",
       "L 56.522857 72.44034 \n",
       "L 57.127217 72.982871 \n",
       "L 57.731932 73.538727 \n",
       "L 58.33695 74.105575 \n",
       "L 58.942217 74.680672 \n",
       "L 59.547672 75.26086 \n",
       "L 60.15326 75.842582 \n",
       "L 60.758927 76.421899 \n",
       "L 61.364617 76.99451 \n",
       "L 61.970283 77.555797 \n",
       "L 62.575882 78.100867 \n",
       "L 63.181373 78.624606 \n",
       "L 63.78673 79.121744 \n",
       "L 64.391931 79.586933 \n",
       "L 64.996966 80.014819 \n",
       "L 65.601835 80.400132 \n",
       "L 66.20655 80.737777 \n",
       "L 66.811134 81.022926 \n",
       "L 67.415625 81.251112 \n",
       "L 68.020071 81.418328 \n",
       "L 68.62453 81.521111 \n",
       "L 69.229075 81.556632 \n",
       "L 69.833787 81.522776 \n",
       "L 70.438758 81.418208 \n",
       "L 71.044084 81.242439 \n",
       "L 71.649869 80.995864 \n",
       "L 72.256221 80.679801 \n",
       "L 72.863249 80.296505 \n",
       "L 73.471063 79.849167 \n",
       "L 74.079768 79.341896 \n",
       "L 74.689465 78.779689 \n",
       "L 75.300248 78.16837 \n",
       "L 75.912202 77.514526 \n",
       "L 76.525399 76.825418 \n",
       "L 77.139898 76.108881 \n",
       "L 77.755747 75.373209 \n",
       "L 78.372974 74.627026 \n",
       "L 78.991591 73.879163 \n",
       "L 79.611595 73.138506 \n",
       "L 80.232966 72.413859 \n",
       "L 80.855667 71.713804 \n",
       "L 81.479644 71.04656 \n",
       "L 82.104831 70.419857 \n",
       "L 82.731148 69.840807 \n",
       "L 83.358505 69.315803 \n",
       "L 83.9868 68.850428 \n",
       "L 84.615927 68.449371 \n",
       "L 85.245776 68.116379 \n",
       "L 85.876234 67.854219 \n",
       "L 86.50719 67.664658 \n",
       "L 87.138534 67.548471 \n",
       "L 87.770161 67.505468 \n",
       "L 88.401978 67.53453 \n",
       "L 89.033894 67.633671 \n",
       "L 89.665836 67.800112 \n",
       "L 90.297735 68.030362 \n",
       "L 90.92954 68.320321 \n",
       "L 91.561213 68.665374 \n",
       "L 92.192723 69.060496 \n",
       "L 92.824061 69.500367 \n",
       "L 93.455223 69.979466 \n",
       "L 94.08622 70.49218 \n",
       "L 94.717073 71.032897 \n",
       "L 95.347817 71.596097 \n",
       "L 95.978486 72.176422 \n",
       "L 96.60913 72.768761 \n",
       "L 97.239803 73.368298 \n",
       "L 97.870558 73.970558 \n",
       "L 98.501456 74.571455 \n",
       "L 99.13256 75.16731 \n",
       "L 99.763929 75.754865 \n",
       "L 100.395624 76.331305 \n",
       "L 101.027705 76.894236 \n",
       "L 101.660225 77.441691 \n",
       "L 102.293239 77.972104 \n",
       "L 102.926795 78.484296 \n",
       "L 103.560933 78.977442 \n",
       "L 104.195695 79.451046 \n",
       "L 104.831116 79.904912 \n",
       "L 105.467221 80.339098 \n",
       "L 106.104035 80.753898 \n",
       "L 106.741577 81.149799 \n",
       "L 107.379864 81.527451 \n",
       "L 108.018902 81.887633 \n",
       "L 108.658698 82.231228 \n",
       "L 109.299258 82.559194 \n",
       "L 109.940577 82.872538 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 51.797125 63.652616 \n",
       "L 52.389695 63.926895 \n",
       "L 52.982884 64.213557 \n",
       "L 53.576688 64.513436 \n",
       "L 54.171107 64.827325 \n",
       "L 54.766136 65.155955 \n",
       "L 55.361765 65.499966 \n",
       "L 55.957984 65.859892 \n",
       "L 56.554778 66.236129 \n",
       "L 57.152132 66.628913 \n",
       "L 57.75002 67.03828 \n",
       "L 58.348419 67.464056 \n",
       "L 58.947304 67.905818 \n",
       "L 59.546638 68.36287 \n",
       "L 60.14639 68.834223 \n",
       "L 60.746525 69.318575 \n",
       "L 61.346999 69.814286 \n",
       "L 61.947775 70.319378 \n",
       "L 62.548813 70.831525 \n",
       "L 63.150069 71.348045 \n",
       "L 63.751506 71.865921 \n",
       "L 64.353087 72.381805 \n",
       "L 64.954773 72.892042 \n",
       "L 65.556538 73.392709 \n",
       "L 66.158355 73.879643 \n",
       "L 66.760202 74.348495 \n",
       "L 67.362068 74.794784 \n",
       "L 67.96395 75.213955 \n",
       "L 68.565849 75.601458 \n",
       "L 69.167777 75.952801 \n",
       "L 69.769758 76.263652 \n",
       "L 70.371821 76.529901 \n",
       "L 70.97401 76.747749 \n",
       "L 71.576373 76.913784 \n",
       "L 72.178968 77.025065 \n",
       "L 72.781865 77.079191 \n",
       "L 73.385135 77.074369 \n",
       "L 73.98886 77.009475 \n",
       "L 74.593122 76.884101 \n",
       "L 75.198008 76.698602 \n",
       "L 75.803607 76.454116 \n",
       "L 76.410006 76.152578 \n",
       "L 77.017288 75.796722 \n",
       "L 77.625536 75.390062 \n",
       "L 78.23482 74.936867 \n",
       "L 78.845208 74.442105 \n",
       "L 79.456756 73.911392 \n",
       "L 80.069507 73.350911 \n",
       "L 80.683494 72.767328 \n",
       "L 81.298736 72.167694 \n",
       "L 81.91524 71.559334 \n",
       "L 82.532992 70.949742 \n",
       "L 83.15197 70.346451 \n",
       "L 83.772136 69.756921 \n",
       "L 84.39344 69.188413 \n",
       "L 85.015818 68.647879 \n",
       "L 85.639195 68.141848 \n",
       "L 86.263488 67.676325 \n",
       "L 86.888607 67.256702 \n",
       "L 87.514452 66.88768 \n",
       "L 88.140925 66.573207 \n",
       "L 88.767924 66.316433 \n",
       "L 89.395348 66.119674 \n",
       "L 90.023098 65.98441 \n",
       "L 90.651083 65.911281 \n",
       "L 91.279214 65.900112 \n",
       "L 91.907415 65.949945 \n",
       "L 92.535614 66.059093 \n",
       "L 93.163757 66.225202 \n",
       "L 93.791795 66.44532 \n",
       "L 94.419693 66.715976 \n",
       "L 95.047429 67.03327 \n",
       "L 95.674992 67.392956 \n",
       "L 96.302383 67.790539 \n",
       "L 96.929614 68.221351 \n",
       "L 97.556706 68.680651 \n",
       "L 98.18369 69.163695 \n",
       "L 98.810607 69.665815 \n",
       "L 99.437499 70.182483 \n",
       "L 100.06442 70.709373 \n",
       "L 100.691426 71.242411 \n",
       "L 101.318571 71.777809 \n",
       "L 101.945918 72.312108 \n",
       "L 102.573528 72.842191 \n",
       "L 103.201457 73.365298 \n",
       "L 103.829763 73.87904 \n",
       "L 104.458504 74.38139 \n",
       "L 105.087727 74.870675 \n",
       "L 105.717483 75.345565 \n",
       "L 106.347815 75.805059 \n",
       "L 106.97876 76.248452 \n",
       "L 107.610354 76.675323 \n",
       "L 108.242625 77.085499 \n",
       "L 108.875597 77.479029 \n",
       "L 109.509289 77.856158 \n",
       "L 110.143716 78.217295 \n",
       "L 110.778891 78.562991 \n",
       "L 111.414818 78.893903 \n",
       "L 112.051501 79.210778 \n",
       "L 112.688943 79.514428 \n",
       "L 113.327136 79.805702 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 55.421126 60.475328 \n",
       "L 56.010202 60.706848 \n",
       "L 56.599816 60.946052 \n",
       "L 57.189963 61.193455 \n",
       "L 57.780643 61.449548 \n",
       "L 58.371853 61.714784 \n",
       "L 58.963583 61.989559 \n",
       "L 59.555828 62.274207 \n",
       "L 60.148579 62.568974 \n",
       "L 60.741826 62.87401 \n",
       "L 61.335553 63.189341 \n",
       "L 61.929747 63.514864 \n",
       "L 62.524394 63.850323 \n",
       "L 63.119471 64.195292 \n",
       "L 63.714962 64.549168 \n",
       "L 64.310847 64.911149 \n",
       "L 64.907104 65.280228 \n",
       "L 65.503712 65.65519 \n",
       "L 66.100651 66.034599 \n",
       "L 66.697899 66.416801 \n",
       "L 67.295437 66.799935 \n",
       "L 67.893251 67.181932 \n",
       "L 68.491321 67.560532 \n",
       "L 69.089636 67.933307 \n",
       "L 69.688191 68.297679 \n",
       "L 70.286976 68.650956 \n",
       "L 70.885994 68.990358 \n",
       "L 71.485249 69.313061 \n",
       "L 72.084752 69.616241 \n",
       "L 72.684515 69.897113 \n",
       "L 73.284562 70.152989 \n",
       "L 73.884918 70.381317 \n",
       "L 74.485616 70.579743 \n",
       "L 75.086691 70.746154 \n",
       "L 75.688185 70.878728 \n",
       "L 76.290144 70.975985 \n",
       "L 76.892615 71.036821 \n",
       "L 77.495651 71.060553 \n",
       "L 78.099304 71.046946 \n",
       "L 78.703626 70.996239 \n",
       "L 79.30867 70.909158 \n",
       "L 79.914487 70.786928 \n",
       "L 80.521123 70.631267 \n",
       "L 81.128623 70.44438 \n",
       "L 81.737024 70.228933 \n",
       "L 82.346358 69.988026 \n",
       "L 82.956651 69.725154 \n",
       "L 83.567919 69.44416 \n",
       "L 84.18017 69.149174 \n",
       "L 84.793404 68.844562 \n",
       "L 85.407614 68.534849 \n",
       "L 86.022778 68.224655 \n",
       "L 86.638871 67.918616 \n",
       "L 87.255856 67.621315 \n",
       "L 87.873691 67.337204 \n",
       "L 88.492326 67.070538 \n",
       "L 89.111704 66.825304 \n",
       "L 89.731765 66.605166 \n",
       "L 90.352445 66.413403 \n",
       "L 90.973675 66.25287 \n",
       "L 91.595391 66.125958 \n",
       "L 92.217524 66.03457 \n",
       "L 92.84001 65.980101 \n",
       "L 93.462788 65.963429 \n",
       "L 94.085799 65.984929 \n",
       "L 94.708992 66.044474 \n",
       "L 95.332321 66.141467 \n",
       "L 95.955747 66.274865 \n",
       "L 96.579239 66.443219 \n",
       "L 97.202774 66.644721 \n",
       "L 97.826336 66.877246 \n",
       "L 98.44992 67.138409 \n",
       "L 99.073523 67.425614 \n",
       "L 99.697156 67.736115 \n",
       "L 100.320832 68.067058 \n",
       "L 100.944573 68.415547 \n",
       "L 101.568406 68.778679 \n",
       "L 102.192364 69.1536 \n",
       "L 102.816478 69.537538 \n",
       "L 103.44079 69.927844 \n",
       "L 104.065341 70.322021 \n",
       "L 104.69017 70.717747 \n",
       "L 105.31532 71.112897 \n",
       "L 105.940837 71.505559 \n",
       "L 106.566755 71.894035 \n",
       "L 107.193118 72.276852 \n",
       "L 107.819963 72.652765 \n",
       "L 108.447322 73.020739 \n",
       "L 109.075229 73.379954 \n",
       "L 109.703712 73.729788 \n",
       "L 110.332795 74.069805 \n",
       "L 110.9625 74.399739 \n",
       "L 111.592847 74.719482 \n",
       "L 112.223849 75.029058 \n",
       "L 112.855518 75.328614 \n",
       "L 113.487863 75.6184 \n",
       "L 114.120893 75.89875 \n",
       "L 114.754606 76.170066 \n",
       "L 115.389006 76.432806 \n",
       "L 116.024093 76.687468 \n",
       "L 116.659862 76.934572 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 58.997348 57.368759 \n",
       "L 59.583225 57.562135 \n",
       "L 60.169589 57.759022 \n",
       "L 60.756435 57.959651 \n",
       "L 61.343763 58.164241 \n",
       "L 61.931573 58.372998 \n",
       "L 62.519859 58.586098 \n",
       "L 63.10862 58.803693 \n",
       "L 63.69785 59.025895 \n",
       "L 64.287548 59.252771 \n",
       "L 64.877703 59.484336 \n",
       "L 65.468313 59.720544 \n",
       "L 66.059371 59.961284 \n",
       "L 66.650867 60.206366 \n",
       "L 67.242795 60.455522 \n",
       "L 67.83515 60.708395 \n",
       "L 68.42792 60.964536 \n",
       "L 69.0211 61.223399 \n",
       "L 69.614682 61.484345 \n",
       "L 70.208658 61.746633 \n",
       "L 70.803023 62.009427 \n",
       "L 71.397774 62.271801 \n",
       "L 71.992903 62.53274 \n",
       "L 72.58841 62.791154 \n",
       "L 73.184296 63.045886 \n",
       "L 73.780559 63.295723 \n",
       "L 74.377204 63.539415 \n",
       "L 74.974239 63.775695 \n",
       "L 75.571669 64.003288 \n",
       "L 76.169505 64.220942 \n",
       "L 76.767761 64.427446 \n",
       "L 77.366451 64.621654 \n",
       "L 77.965593 64.802504 \n",
       "L 78.565207 64.969047 \n",
       "L 79.165311 65.120465 \n",
       "L 79.765931 65.256095 \n",
       "L 80.367087 65.375444 \n",
       "L 80.968805 65.478211 \n",
       "L 81.571109 65.564296 \n",
       "L 82.174022 65.633814 \n",
       "L 82.777567 65.687101 \n",
       "L 83.381765 65.724718 \n",
       "L 83.986635 65.747447 \n",
       "L 84.592195 65.756288 \n",
       "L 85.198459 65.752452 \n",
       "L 85.805436 65.737339 \n",
       "L 86.413135 65.712526 \n",
       "L 87.021558 65.679746 \n",
       "L 87.630705 65.640858 \n",
       "L 88.24057 65.597821 \n",
       "L 88.851145 65.552665 \n",
       "L 89.462414 65.507457 \n",
       "L 90.074361 65.464271 \n",
       "L 90.686965 65.42515 \n",
       "L 91.300202 65.392079 \n",
       "L 91.914045 65.366948 \n",
       "L 92.528465 65.35153 \n",
       "L 93.143429 65.347447 \n",
       "L 93.758909 65.356151 \n",
       "L 94.374868 65.378903 \n",
       "L 94.991277 65.416755 \n",
       "L 95.608103 65.470545 \n",
       "L 96.225316 65.540883 \n",
       "L 96.84289 65.628152 \n",
       "L 97.460798 65.732509 \n",
       "L 98.079017 65.853892 \n",
       "L 98.697529 65.992029 \n",
       "L 99.316317 66.146453 \n",
       "L 99.935372 66.316518 \n",
       "L 100.554685 66.501417 \n",
       "L 101.17425 66.700207 \n",
       "L 101.794069 66.911826 \n",
       "L 102.414145 67.135122 \n",
       "L 103.034486 67.368875 \n",
       "L 103.655102 67.611818 \n",
       "L 104.276004 67.862663 \n",
       "L 104.89721 68.120121 \n",
       "L 105.518738 68.382923 \n",
       "L 106.140604 68.649836 \n",
       "L 106.762831 68.919682 \n",
       "L 107.385441 69.191349 \n",
       "L 108.008452 69.463803 \n",
       "L 108.631889 69.736095 \n",
       "L 109.255773 70.007373 \n",
       "L 109.880122 70.276878 \n",
       "L 110.504956 70.543954 \n",
       "L 111.130297 70.808042 \n",
       "L 111.756157 71.068679 \n",
       "L 112.382554 71.325496 \n",
       "L 113.009502 71.578216 \n",
       "L 113.637011 71.826641 \n",
       "L 114.265093 72.070651 \n",
       "L 114.893758 72.310196 \n",
       "L 115.523009 72.545284 \n",
       "L 116.152854 72.775981 \n",
       "L 116.783297 73.002396 \n",
       "L 117.414342 73.224679 \n",
       "L 118.045987 73.443007 \n",
       "L 118.678234 73.657584 \n",
       "L 119.311084 73.868632 \n",
       "L 119.944532 74.076384 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 62.527461 54.41079 \n",
       "L 63.110492 54.58233 \n",
       "L 63.693988 54.755096 \n",
       "L 64.277943 54.929163 \n",
       "L 64.862362 55.104603 \n",
       "L 65.447243 55.281483 \n",
       "L 66.032584 55.45986 \n",
       "L 66.618384 55.639784 \n",
       "L 67.204644 55.821291 \n",
       "L 67.791363 56.004405 \n",
       "L 68.378537 56.189129 \n",
       "L 68.966166 56.37545 \n",
       "L 69.554249 56.563332 \n",
       "L 70.142783 56.752714 \n",
       "L 70.731765 56.943508 \n",
       "L 71.321197 57.1356 \n",
       "L 71.911074 57.328843 \n",
       "L 72.501394 57.52306 \n",
       "L 73.092159 57.718046 \n",
       "L 73.683364 57.913557 \n",
       "L 74.27501 58.109322 \n",
       "L 74.867098 58.305042 \n",
       "L 75.459624 58.500384 \n",
       "L 76.052591 58.694995 \n",
       "L 76.646002 58.888499 \n",
       "L 77.239855 59.0805 \n",
       "L 77.834155 59.27059 \n",
       "L 78.428905 59.458358 \n",
       "L 79.024109 59.643388 \n",
       "L 79.619771 59.825272 \n",
       "L 80.215898 60.003616 \n",
       "L 80.812495 60.178045 \n",
       "L 81.40957 60.348215 \n",
       "L 82.00713 60.513815 \n",
       "L 82.605181 60.674581 \n",
       "L 83.203735 60.830296 \n",
       "L 83.802796 60.980801 \n",
       "L 84.402376 61.125999 \n",
       "L 85.002482 61.265858 \n",
       "L 85.603121 61.400418 \n",
       "L 86.204302 61.529789 \n",
       "L 86.80603 61.654155 \n",
       "L 87.408312 61.773775 \n",
       "L 88.011153 61.888975 \n",
       "L 88.614557 62.000151 \n",
       "L 89.218526 62.107761 \n",
       "L 89.823061 62.212319 \n",
       "L 90.428163 62.314392 \n",
       "L 91.033831 62.414583 \n",
       "L 91.64006 62.513533 \n",
       "L 92.246848 62.6119 \n",
       "L 92.854186 62.710356 \n",
       "L 93.46207 62.809575 \n",
       "L 94.07049 62.910219 \n",
       "L 94.679438 63.012931 \n",
       "L 95.288903 63.118325 \n",
       "L 95.898876 63.226973 \n",
       "L 96.509344 63.339399 \n",
       "L 97.120299 63.456072 \n",
       "L 97.731726 63.5774 \n",
       "L 98.343617 63.703721 \n",
       "L 98.95596 63.835305 \n",
       "L 99.568747 63.972348 \n",
       "L 100.181967 64.114971 \n",
       "L 100.795614 64.263226 \n",
       "L 101.409679 64.417089 \n",
       "L 102.024159 64.576474 \n",
       "L 102.639048 64.741227 \n",
       "L 103.254345 64.91114 \n",
       "L 103.870047 65.085951 \n",
       "L 104.486153 65.265355 \n",
       "L 105.102668 65.449008 \n",
       "L 105.71959 65.636537 \n",
       "L 106.336926 65.827548 \n",
       "L 106.95468 66.021631 \n",
       "L 107.572856 66.21837 \n",
       "L 108.191463 66.417347 \n",
       "L 108.810508 66.618152 \n",
       "L 109.429996 66.820385 \n",
       "L 110.049936 67.023665 \n",
       "L 110.67034 67.227633 \n",
       "L 111.29121 67.431952 \n",
       "L 111.912558 67.636315 \n",
       "L 112.534393 67.840448 \n",
       "L 113.156718 68.044102 \n",
       "L 113.779544 68.247065 \n",
       "L 114.402879 68.449157 \n",
       "L 115.026725 68.650225 \n",
       "L 115.65109 68.850151 \n",
       "L 116.27598 69.048845 \n",
       "L 116.901398 69.246241 \n",
       "L 117.527347 69.442301 \n",
       "L 118.153834 69.637009 \n",
       "L 118.780857 69.830366 \n",
       "L 119.408421 70.022394 \n",
       "L 120.036526 70.213129 \n",
       "L 120.665177 70.402619 \n",
       "L 121.29437 70.59092 \n",
       "L 121.924106 70.778101 \n",
       "L 122.554389 70.964232 \n",
       "L 123.185214 71.149389 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 66.011299 51.574547 \n",
       "L 66.59168 51.736694 \n",
       "L 67.172516 51.899225 \n",
       "L 67.753804 52.062158 \n",
       "L 68.335547 52.22551 \n",
       "L 68.917746 52.389296 \n",
       "L 69.500398 52.553532 \n",
       "L 70.083505 52.718227 \n",
       "L 70.667068 52.883391 \n",
       "L 71.251087 53.049031 \n",
       "L 71.83556 53.215146 \n",
       "L 72.420488 53.381734 \n",
       "L 73.005874 53.548786 \n",
       "L 73.591713 53.716288 \n",
       "L 74.178008 53.88422 \n",
       "L 74.76476 54.052554 \n",
       "L 75.351966 54.221257 \n",
       "L 75.939627 54.390286 \n",
       "L 76.527746 54.559594 \n",
       "L 77.11632 54.729122 \n",
       "L 77.705351 54.898807 \n",
       "L 78.29484 55.06858 \n",
       "L 78.884785 55.23836 \n",
       "L 79.475189 55.408066 \n",
       "L 80.066054 55.577607 \n",
       "L 80.657378 55.746892 \n",
       "L 81.249163 55.915823 \n",
       "L 81.841413 56.084305 \n",
       "L 82.434127 56.252238 \n",
       "L 83.027308 56.419526 \n",
       "L 83.620957 56.586077 \n",
       "L 84.215077 56.751803 \n",
       "L 84.809671 56.916621 \n",
       "L 85.40474 57.080459 \n",
       "L 86.000286 57.243255 \n",
       "L 86.596313 57.404957 \n",
       "L 87.192822 57.565528 \n",
       "L 87.789817 57.724946 \n",
       "L 88.387299 57.883203 \n",
       "L 88.985271 58.040308 \n",
       "L 89.583735 58.196289 \n",
       "L 90.182693 58.351188 \n",
       "L 90.782147 58.505068 \n",
       "L 91.382097 58.658005 \n",
       "L 91.982546 58.810092 \n",
       "L 92.583493 58.96144 \n",
       "L 93.184939 59.112168 \n",
       "L 93.786885 59.262412 \n",
       "L 94.389329 59.412313 \n",
       "L 94.992272 59.562022 \n",
       "L 95.595714 59.711697 \n",
       "L 96.19965 59.861494 \n",
       "L 96.80408 60.011573 \n",
       "L 97.409003 60.16209 \n",
       "L 98.014417 60.313197 \n",
       "L 98.620318 60.465038 \n",
       "L 99.226705 60.617749 \n",
       "L 99.833576 60.771452 \n",
       "L 100.440927 60.926259 \n",
       "L 101.048757 61.082265 \n",
       "L 101.657063 61.239549 \n",
       "L 102.265843 61.398177 \n",
       "L 102.875094 61.558192 \n",
       "L 103.484817 61.719625 \n",
       "L 104.095009 61.882488 \n",
       "L 104.705667 62.046774 \n",
       "L 105.316793 62.212463 \n",
       "L 105.928385 62.37952 \n",
       "L 106.540444 62.547895 \n",
       "L 107.15297 62.717527 \n",
       "L 107.765963 62.888343 \n",
       "L 108.379424 63.060264 \n",
       "L 108.993355 63.233201 \n",
       "L 109.607757 63.407063 \n",
       "L 110.222632 63.581751 \n",
       "L 110.837982 63.757169 \n",
       "L 111.453809 63.933218 \n",
       "L 112.070117 64.109802 \n",
       "L 112.686905 64.286826 \n",
       "L 113.304178 64.464201 \n",
       "L 113.921941 64.641843 \n",
       "L 114.540191 64.819672 \n",
       "L 115.158934 64.997616 \n",
       "L 115.778173 65.17561 \n",
       "L 116.397908 65.353597 \n",
       "L 117.018142 65.531525 \n",
       "L 117.638881 65.709354 \n",
       "L 118.260121 65.887046 \n",
       "L 118.881866 66.064574 \n",
       "L 119.504121 66.241918 \n",
       "L 120.126883 66.419061 \n",
       "L 120.750156 66.595994 \n",
       "L 121.373941 66.772715 \n",
       "L 121.998238 66.949222 \n",
       "L 122.623048 67.125523 \n",
       "L 123.248373 67.301624 \n",
       "L 123.874215 67.477539 \n",
       "L 124.50057 67.653279 \n",
       "L 125.127442 67.828862 \n",
       "L 125.754832 68.004304 \n",
       "L 126.382738 68.179622 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "   </g>\n",
       "   <g id=\"Line3DCollection_5\">\n",
       "    <path d=\"M 27.324045 -1 \n",
       "L 27.513159 -0.067972 \n",
       "L 28.038884 2.472454 \n",
       "L 28.559414 4.937136 \n",
       "L 29.074866 7.327049 \n",
       "L 29.585348 9.64312 \n",
       "L 30.090971 11.886252 \n",
       "L 30.591837 14.057303 \n",
       "L 31.088051 16.157118 \n",
       "L 31.579713 18.186511 \n",
       "L 32.066921 20.146262 \n",
       "L 32.549773 22.037144 \n",
       "L 33.028363 23.859879 \n",
       "L 33.502782 25.615184 \n",
       "L 33.973121 27.303746 \n",
       "L 34.439471 28.926221 \n",
       "L 34.901915 30.483255 \n",
       "L 35.360539 31.975464 \n",
       "L 35.815429 33.403441 \n",
       "L 36.266663 34.767756 \n",
       "L 36.714324 36.068969 \n",
       "L 37.158489 37.307605 \n",
       "L 37.599235 38.484178 \n",
       "L 38.036637 39.59918 \n",
       "L 38.470771 40.653085 \n",
       "L 38.901708 41.646345 \n",
       "L 39.329521 42.579393 \n",
       "L 39.754279 43.452644 \n",
       "L 40.176051 44.266504 \n",
       "L 40.594906 45.021345 \n",
       "L 41.010909 45.717536 \n",
       "L 41.424126 46.355419 \n",
       "L 41.834623 46.935324 \n",
       "L 42.24246 47.457567 \n",
       "L 42.647703 47.922444 \n",
       "L 43.050411 48.330234 \n",
       "L 43.450645 48.681199 \n",
       "L 43.848465 48.975591 \n",
       "L 44.243929 49.213645 \n",
       "L 44.637094 49.395575 \n",
       "L 45.028019 49.521588 \n",
       "L 45.416758 49.591869 \n",
       "L 45.803367 49.606594 \n",
       "L 46.187901 49.565919 \n",
       "L 46.570414 49.469991 \n",
       "L 46.950958 49.318938 \n",
       "L 47.329587 49.112875 \n",
       "L 47.706352 48.851906 \n",
       "L 48.081304 48.536117 \n",
       "L 48.454494 48.165583 \n",
       "L 48.825971 47.740362 \n",
       "L 49.195786 47.2605 \n",
       "L 49.563988 46.72603 \n",
       "L 49.930624 46.136969 \n",
       "L 50.295742 45.493323 \n",
       "L 50.659392 44.795083 \n",
       "L 51.021618 44.042227 \n",
       "L 51.382469 43.234717 \n",
       "L 51.741989 42.372507 \n",
       "L 52.100226 41.455531 \n",
       "L 52.457224 40.483712 \n",
       "L 52.813029 39.456959 \n",
       "L 53.167685 38.375172 \n",
       "L 53.521237 37.238232 \n",
       "L 53.873729 36.046006 \n",
       "L 54.225206 34.798348 \n",
       "L 54.57571 33.495101 \n",
       "L 54.925284 32.136092 \n",
       "L 55.273974 30.721138 \n",
       "L 55.62182 29.250033 \n",
       "L 55.968868 27.722563 \n",
       "L 56.315158 26.138501 \n",
       "L 56.660733 24.497605 \n",
       "L 57.005636 22.799616 \n",
       "L 57.34991 21.044258 \n",
       "L 57.693596 19.231247 \n",
       "L 58.036736 17.360284 \n",
       "L 58.379374 15.431044 \n",
       "L 58.721549 13.443201 \n",
       "L 59.063305 11.396408 \n",
       "L 59.404685 9.2903 \n",
       "L 59.745728 7.124498 \n",
       "L 60.086477 4.898608 \n",
       "L 60.426977 2.61222 \n",
       "L 60.767266 0.2649 \n",
       "L 60.945879 -1 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #800080; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 32.42381 7.021612 \n",
       "L 32.946141 9.799662 \n",
       "L 33.463313 12.499689 \n",
       "L 33.975438 15.122738 \n",
       "L 34.48263 17.669835 \n",
       "L 34.985001 20.141974 \n",
       "L 35.482656 22.540103 \n",
       "L 35.9757 24.865148 \n",
       "L 36.464238 27.118014 \n",
       "L 36.948371 29.299576 \n",
       "L 37.428194 31.410664 \n",
       "L 37.903804 33.452095 \n",
       "L 38.375298 35.424664 \n",
       "L 38.842765 37.329121 \n",
       "L 39.306296 39.166216 \n",
       "L 39.76598 40.936655 \n",
       "L 40.221901 42.641115 \n",
       "L 40.674146 44.280275 \n",
       "L 41.122798 45.854779 \n",
       "L 41.567937 47.365234 \n",
       "L 42.009643 48.812246 \n",
       "L 42.447995 50.196391 \n",
       "L 42.883069 51.51822 \n",
       "L 43.314939 52.778275 \n",
       "L 43.743682 53.977066 \n",
       "L 44.169367 55.115088 \n",
       "L 44.592067 56.192821 \n",
       "L 45.011852 57.210717 \n",
       "L 45.42879 58.169222 \n",
       "L 45.842948 59.06875 \n",
       "L 46.254393 59.90971 \n",
       "L 46.66319 60.692485 \n",
       "L 47.069403 61.417443 \n",
       "L 47.473096 62.084935 \n",
       "L 47.874329 62.695297 \n",
       "L 48.273165 63.248849 \n",
       "L 48.669662 63.74589 \n",
       "L 49.063882 64.186711 \n",
       "L 49.455881 64.57158 \n",
       "L 49.845718 64.900754 \n",
       "L 50.233449 65.174473 \n",
       "L 50.619129 65.392962 \n",
       "L 51.002814 65.556432 \n",
       "L 51.384559 65.665079 \n",
       "L 51.764417 65.719084 \n",
       "L 52.14244 65.718614 \n",
       "L 52.518682 65.663822 \n",
       "L 52.893193 65.554846 \n",
       "L 53.266025 65.391812 \n",
       "L 53.637229 65.17483 \n",
       "L 54.006855 64.903995 \n",
       "L 54.37495 64.579395 \n",
       "L 54.741565 64.201096 \n",
       "L 55.106748 63.769155 \n",
       "L 55.470546 63.283616 \n",
       "L 55.833008 62.744507 \n",
       "L 56.194179 62.151846 \n",
       "L 56.554108 61.505634 \n",
       "L 56.912839 60.805861 \n",
       "L 57.270418 60.052504 \n",
       "L 57.626892 59.245525 \n",
       "L 57.982304 58.384875 \n",
       "L 58.336701 57.470489 \n",
       "L 58.690126 56.502292 \n",
       "L 59.042625 55.480194 \n",
       "L 59.394239 54.404093 \n",
       "L 59.745014 53.273868 \n",
       "L 60.094993 52.089394 \n",
       "L 60.444219 50.850527 \n",
       "L 60.792736 49.557108 \n",
       "L 61.140585 48.208969 \n",
       "L 61.487812 46.805925 \n",
       "L 61.834456 45.347783 \n",
       "L 62.180562 43.834325 \n",
       "L 62.526172 42.265332 \n",
       "L 62.871328 40.640561 \n",
       "L 63.216072 38.959763 \n",
       "L 63.560448 37.222665 \n",
       "L 63.904496 35.428989 \n",
       "L 64.248259 33.578442 \n",
       "L 64.591781 31.670705 \n",
       "L 64.935101 29.705461 \n",
       "L 65.278263 27.682368 \n",
       "L 65.621311 25.60106 \n",
       "L 65.964284 23.461181 \n",
       "L 66.307227 21.262342 \n",
       "L 66.650182 19.004124 \n",
       "L 66.993191 16.686124 \n",
       "L 67.336298 14.30791 \n",
       "L 67.679547 11.869016 \n",
       "L 68.022978 9.368992 \n",
       "L 68.366637 6.807342 \n",
       "L 68.710569 4.18356 \n",
       "L 69.054814 1.49714 \n",
       "L 69.367778 -1 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #800080; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 39.210436 18.683938 \n",
       "L 39.710885 21.418924 \n",
       "L 40.206624 24.076887 \n",
       "L 40.697758 26.658859 \n",
       "L 41.184394 29.165848 \n",
       "L 41.666638 31.598834 \n",
       "L 42.144588 33.958747 \n",
       "L 42.618342 36.246502 \n",
       "L 43.087999 38.46298 \n",
       "L 43.553654 40.609051 \n",
       "L 44.015397 42.685522 \n",
       "L 44.473319 44.693208 \n",
       "L 44.927511 46.632879 \n",
       "L 45.378056 48.505283 \n",
       "L 45.825042 50.311146 \n",
       "L 46.268551 52.051172 \n",
       "L 46.708663 53.726024 \n",
       "L 47.14546 55.336368 \n",
       "L 47.579021 56.882841 \n",
       "L 48.00942 58.366033 \n",
       "L 48.436734 59.786547 \n",
       "L 48.861038 61.144948 \n",
       "L 49.282402 62.441773 \n",
       "L 49.700899 63.677556 \n",
       "L 50.1166 64.852804 \n",
       "L 50.529571 65.967998 \n",
       "L 50.939882 67.023611 \n",
       "L 51.347598 68.020092 \n",
       "L 51.752784 68.957871 \n",
       "L 52.155505 69.837362 \n",
       "L 52.555824 70.658962 \n",
       "L 52.953803 71.42305 \n",
       "L 53.349503 72.129987 \n",
       "L 53.742984 72.780119 \n",
       "L 54.134306 73.373774 \n",
       "L 54.523526 73.911267 \n",
       "L 54.910703 74.392893 \n",
       "L 55.295893 74.818934 \n",
       "L 55.679153 75.189657 \n",
       "L 56.060536 75.505312 \n",
       "L 56.440098 75.766135 \n",
       "L 56.817893 75.972346 \n",
       "L 57.193972 76.124153 \n",
       "L 57.56839 76.221748 \n",
       "L 57.941197 76.265307 \n",
       "L 58.312444 76.254995 \n",
       "L 58.682183 76.19096 \n",
       "L 59.050463 76.073339 \n",
       "L 59.417333 75.902252 \n",
       "L 59.782843 75.67781 \n",
       "L 60.147042 75.400104 \n",
       "L 60.509975 75.069217 \n",
       "L 60.871693 74.685216 \n",
       "L 61.23224 74.248157 \n",
       "L 61.591665 73.758078 \n",
       "L 61.950014 73.21501 \n",
       "L 62.307332 72.618965 \n",
       "L 62.663665 71.969947 \n",
       "L 63.019059 71.267941 \n",
       "L 63.373558 70.512927 \n",
       "L 63.727207 69.704864 \n",
       "L 64.08005 68.843702 \n",
       "L 64.432133 67.929378 \n",
       "L 64.783497 66.961814 \n",
       "L 65.134189 65.940921 \n",
       "L 65.484249 64.866598 \n",
       "L 65.833723 63.738725 \n",
       "L 66.182653 62.557176 \n",
       "L 66.531083 61.321807 \n",
       "L 66.879055 60.032462 \n",
       "L 67.226612 58.688973 \n",
       "L 67.573797 57.291157 \n",
       "L 67.920652 55.838821 \n",
       "L 68.267221 54.33175 \n",
       "L 68.613545 52.769724 \n",
       "L 68.959667 51.152507 \n",
       "L 69.30563 49.479847 \n",
       "L 69.651477 47.751474 \n",
       "L 69.99725 45.967118 \n",
       "L 70.342991 44.126483 \n",
       "L 70.688744 42.229253 \n",
       "L 71.03455 40.275117 \n",
       "L 71.380454 38.263736 \n",
       "L 71.726498 36.194744 \n",
       "L 72.072725 34.067793 \n",
       "L 72.419179 31.882496 \n",
       "L 72.765904 29.638439 \n",
       "L 73.112941 27.335224 \n",
       "L 73.460337 24.972417 \n",
       "L 73.808135 22.549567 \n",
       "L 74.156379 20.066218 \n",
       "L 74.505114 17.52189 \n",
       "L 74.854386 14.916073 \n",
       "L 75.204238 12.248276 \n",
       "L 75.554718 9.517949 \n",
       "L 75.90587 6.724547 \n",
       "L 76.257743 3.867496 \n",
       "L 76.610382 0.946224 \n",
       "L 76.840747 -1 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #800080; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 45.81064 24.194222 \n",
       "L 46.292058 26.917041 \n",
       "L 46.769167 29.563046 \n",
       "L 47.242069 32.133248 \n",
       "L 47.710863 34.628661 \n",
       "L 48.17565 37.050262 \n",
       "L 48.636522 39.398974 \n",
       "L 49.093574 41.675725 \n",
       "L 49.546898 43.881383 \n",
       "L 49.996583 46.016817 \n",
       "L 50.442717 48.08284 \n",
       "L 50.885385 50.080254 \n",
       "L 51.324673 52.009839 \n",
       "L 51.76066 53.872331 \n",
       "L 52.19343 55.668467 \n",
       "L 52.623061 57.398937 \n",
       "L 53.049629 59.064409 \n",
       "L 53.473211 60.665547 \n",
       "L 53.893884 62.202985 \n",
       "L 54.311717 63.677315 \n",
       "L 54.726784 65.089134 \n",
       "L 55.139157 66.439007 \n",
       "L 55.548902 67.727474 \n",
       "L 55.956089 68.955063 \n",
       "L 56.360786 70.122281 \n",
       "L 56.763055 71.22961 \n",
       "L 57.162964 72.277519 \n",
       "L 57.560576 73.266457 \n",
       "L 57.955953 74.196853 \n",
       "L 58.349155 75.069119 \n",
       "L 58.740246 75.883653 \n",
       "L 59.129282 76.64083 \n",
       "L 59.516325 77.341013 \n",
       "L 59.901431 77.984544 \n",
       "L 60.284657 78.571751 \n",
       "L 60.66606 79.102949 \n",
       "L 61.045695 79.578432 \n",
       "L 61.423618 79.99848 \n",
       "L 61.799882 80.363359 \n",
       "L 62.174539 80.673318 \n",
       "L 62.547644 80.928594 \n",
       "L 62.919248 81.129405 \n",
       "L 63.289402 81.275957 \n",
       "L 63.658159 81.368443 \n",
       "L 64.025566 81.407037 \n",
       "L 64.391675 81.391903 \n",
       "L 64.756535 81.323189 \n",
       "L 65.120195 81.201031 \n",
       "L 65.482702 81.02555 \n",
       "L 65.844105 80.796851 \n",
       "L 66.204452 80.51503 \n",
       "L 66.563788 80.180167 \n",
       "L 66.922161 79.792329 \n",
       "L 67.279616 79.351568 \n",
       "L 67.636201 78.857928 \n",
       "L 67.99196 78.311432 \n",
       "L 68.346939 77.712097 \n",
       "L 68.701183 77.059923 \n",
       "L 69.054736 76.354897 \n",
       "L 69.407644 75.596996 \n",
       "L 69.75995 74.78618 \n",
       "L 70.111698 73.922398 \n",
       "L 70.462933 73.005587 \n",
       "L 70.813698 72.035667 \n",
       "L 71.164037 71.01255 \n",
       "L 71.513992 69.936134 \n",
       "L 71.863609 68.806299 \n",
       "L 72.212928 67.622918 \n",
       "L 72.561995 66.385846 \n",
       "L 72.910852 65.094927 \n",
       "L 73.259541 63.749994 \n",
       "L 73.608106 62.350861 \n",
       "L 73.95659 60.897336 \n",
       "L 74.305037 59.389205 \n",
       "L 74.653488 57.826245 \n",
       "L 75.001987 56.208221 \n",
       "L 75.350577 54.534881 \n",
       "L 75.699303 52.805955 \n",
       "L 76.048205 51.021171 \n",
       "L 76.397329 49.180234 \n",
       "L 76.746718 47.282828 \n",
       "L 77.096415 45.328639 \n",
       "L 77.446464 43.31733 \n",
       "L 77.79691 41.248535 \n",
       "L 78.147797 39.121905 \n",
       "L 78.499168 36.937052 \n",
       "L 78.85107 34.693563 \n",
       "L 79.203547 32.391036 \n",
       "L 79.556644 30.029046 \n",
       "L 79.910407 27.607132 \n",
       "L 80.264882 25.124845 \n",
       "L 80.620115 22.581696 \n",
       "L 80.976153 19.977186 \n",
       "L 81.333043 17.310815 \n",
       "L 81.690831 14.582033 \n",
       "L 82.049567 11.790306 \n",
       "L 82.4093 8.935045 \n",
       "L 82.770076 6.015684 \n",
       "L 83.131945 3.03161 \n",
       "L 83.49496 -0.017815 \n",
       "L 83.609782 -1 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #800080; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 52.313839 23.597901 \n",
       "L 52.77856 26.339314 \n",
       "L 53.23934 29.003296 \n",
       "L 53.696275 31.590892 \n",
       "L 54.149462 34.103112 \n",
       "L 54.598993 36.54095 \n",
       "L 55.044959 38.905343 \n",
       "L 55.48745 41.197229 \n",
       "L 55.926553 43.417492 \n",
       "L 56.362354 45.567014 \n",
       "L 56.794936 47.646609 \n",
       "L 57.22438 49.657101 \n",
       "L 57.65077 51.599272 \n",
       "L 58.074181 53.47387 \n",
       "L 58.494691 55.281639 \n",
       "L 58.912377 57.023281 \n",
       "L 59.327312 58.699481 \n",
       "L 59.739569 60.310895 \n",
       "L 60.149221 61.85817 \n",
       "L 60.556336 63.341916 \n",
       "L 60.960984 64.762727 \n",
       "L 61.363234 66.121177 \n",
       "L 61.76315 67.417814 \n",
       "L 62.160799 68.653172 \n",
       "L 62.556246 69.827764 \n",
       "L 62.949553 70.942079 \n",
       "L 63.340782 71.996591 \n",
       "L 63.729996 72.991755 \n",
       "L 64.117254 73.928004 \n",
       "L 64.502615 74.805756 \n",
       "L 64.886139 75.625414 \n",
       "L 65.267883 76.387357 \n",
       "L 65.647904 77.091952 \n",
       "L 66.026258 77.739547 \n",
       "L 66.403 78.330473 \n",
       "L 66.778186 78.865047 \n",
       "L 67.151868 79.343567 \n",
       "L 67.524101 79.766319 \n",
       "L 67.894937 80.13357 \n",
       "L 68.264427 80.445573 \n",
       "L 68.632624 80.702566 \n",
       "L 68.999577 80.904771 \n",
       "L 69.365338 81.052397 \n",
       "L 69.729957 81.145636 \n",
       "L 70.093481 81.184668 \n",
       "L 70.455961 81.169657 \n",
       "L 70.817445 81.100754 \n",
       "L 71.17798 80.978095 \n",
       "L 71.537614 80.801802 \n",
       "L 71.896395 80.571985 \n",
       "L 72.254369 80.288737 \n",
       "L 72.611581 79.95214 \n",
       "L 72.96808 79.562262 \n",
       "L 73.323909 79.119158 \n",
       "L 73.679116 78.622867 \n",
       "L 74.033745 78.073418 \n",
       "L 74.387841 77.470825 \n",
       "L 74.741448 76.815089 \n",
       "L 75.094613 76.106197 \n",
       "L 75.447379 75.344125 \n",
       "L 75.79979 74.528834 \n",
       "L 76.15189 73.660271 \n",
       "L 76.503724 72.738372 \n",
       "L 76.855335 71.763058 \n",
       "L 77.206767 70.734239 \n",
       "L 77.558064 69.651811 \n",
       "L 77.90927 68.515654 \n",
       "L 78.260427 67.325639 \n",
       "L 78.61158 66.081619 \n",
       "L 78.962773 64.783437 \n",
       "L 79.314047 63.430924 \n",
       "L 79.665449 62.023892 \n",
       "L 80.017021 60.562147 \n",
       "L 80.368807 59.04547 \n",
       "L 80.720851 57.473639 \n",
       "L 81.073196 55.846415 \n",
       "L 81.425888 54.163542 \n",
       "L 81.77897 52.424749 \n",
       "L 82.132486 50.62976 \n",
       "L 82.486481 48.778275 \n",
       "L 82.841001 46.869977 \n",
       "L 83.196089 44.904547 \n",
       "L 83.551792 42.88164 \n",
       "L 83.908155 40.800898 \n",
       "L 84.265222 38.661958 \n",
       "L 84.623041 36.464422 \n",
       "L 84.98166 34.207888 \n",
       "L 85.341121 31.891942 \n",
       "L 85.701475 29.516149 \n",
       "L 86.062769 27.080046 \n",
       "L 86.425049 24.583178 \n",
       "L 86.788365 22.025057 \n",
       "L 87.152766 19.405167 \n",
       "L 87.5183 16.72301 \n",
       "L 87.885017 13.978031 \n",
       "L 88.252967 11.169682 \n",
       "L 88.622202 8.297371 \n",
       "L 88.992772 5.360526 \n",
       "L 89.36473 2.358518 \n",
       "L 89.738129 -0.709279 \n",
       "L 89.772902 -1 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #800080; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 58.808181 16.772175 \n",
       "L 59.258084 19.564143 \n",
       "L 59.704391 22.277235 \n",
       "L 60.147192 24.912506 \n",
       "L 60.58658 27.471015 \n",
       "L 61.022647 29.953781 \n",
       "L 61.455476 32.361767 \n",
       "L 61.885155 34.695941 \n",
       "L 62.311767 36.957203 \n",
       "L 62.735397 39.146473 \n",
       "L 63.156122 41.264581 \n",
       "L 63.574022 43.312376 \n",
       "L 63.989175 45.29066 \n",
       "L 64.401655 47.200211 \n",
       "L 64.811538 49.041784 \n",
       "L 65.218897 50.816106 \n",
       "L 65.623803 52.523876 \n",
       "L 66.026325 54.165778 \n",
       "L 66.426535 55.742474 \n",
       "L 66.824497 57.254582 \n",
       "L 67.220281 58.702724 \n",
       "L 67.613951 60.087485 \n",
       "L 68.005571 61.409427 \n",
       "L 68.395205 62.669101 \n",
       "L 68.782916 63.867034 \n",
       "L 69.168764 65.003729 \n",
       "L 69.552809 66.079671 \n",
       "L 69.935113 67.095328 \n",
       "L 70.315733 68.051147 \n",
       "L 70.694726 68.947556 \n",
       "L 71.072151 69.784969 \n",
       "L 71.448062 70.563776 \n",
       "L 71.822516 71.284354 \n",
       "L 72.195568 71.947059 \n",
       "L 72.567271 72.552234 \n",
       "L 72.937679 73.100203 \n",
       "L 73.306843 73.591272 \n",
       "L 73.674819 74.025735 \n",
       "L 74.041655 74.403868 \n",
       "L 74.407404 74.725928 \n",
       "L 74.772116 74.992162 \n",
       "L 75.13584 75.202798 \n",
       "L 75.498627 75.358051 \n",
       "L 75.860526 75.458118 \n",
       "L 76.221584 75.503184 \n",
       "L 76.581851 75.493418 \n",
       "L 76.941374 75.428975 \n",
       "L 77.300201 75.309995 \n",
       "L 77.658379 75.136605 \n",
       "L 78.015954 74.908917 \n",
       "L 78.372974 74.627027 \n",
       "L 78.729484 74.291021 \n",
       "L 79.08553 73.900968 \n",
       "L 79.441157 73.456925 \n",
       "L 79.796413 72.958935 \n",
       "L 80.151341 72.407025 \n",
       "L 80.505987 71.801212 \n",
       "L 80.860396 71.141497 \n",
       "L 81.214613 70.427868 \n",
       "L 81.568683 69.6603 \n",
       "L 81.922649 68.838755 \n",
       "L 82.276556 67.963179 \n",
       "L 82.63045 67.033507 \n",
       "L 82.984374 66.049658 \n",
       "L 83.338374 65.011541 \n",
       "L 83.692491 63.919052 \n",
       "L 84.046773 62.772064 \n",
       "L 84.401261 61.570451 \n",
       "L 84.756003 60.314061 \n",
       "L 85.111041 59.002733 \n",
       "L 85.46642 57.636295 \n",
       "L 85.822186 56.214555 \n",
       "L 86.178382 54.737315 \n",
       "L 86.535054 53.204353 \n",
       "L 86.892247 51.615441 \n",
       "L 87.250006 49.970334 \n",
       "L 87.608376 48.268771 \n",
       "L 87.967404 46.510475 \n",
       "L 88.327134 44.695165 \n",
       "L 88.687614 42.822534 \n",
       "L 89.04889 40.892256 \n",
       "L 89.411008 38.904006 \n",
       "L 89.774016 36.857436 \n",
       "L 90.137961 34.75217 \n",
       "L 90.50289 32.587843 \n",
       "L 90.868851 30.364052 \n",
       "L 91.235895 28.080378 \n",
       "L 91.604068 25.7364 \n",
       "L 91.973421 23.331672 \n",
       "L 92.344004 20.865727 \n",
       "L 92.715866 18.33809 \n",
       "L 93.089058 15.748267 \n",
       "L 93.463634 13.095724 \n",
       "L 93.839642 10.379962 \n",
       "L 94.217137 7.600401 \n",
       "L 94.596171 4.756488 \n",
       "L 94.9768 1.847615 \n",
       "L 95.342778 -1 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #800080; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 65.384627 3.414921 \n",
       "L 65.821162 6.292107 \n",
       "L 66.254433 9.088043 \n",
       "L 66.684526 11.803835 \n",
       "L 67.111531 14.440589 \n",
       "L 67.535536 16.999371 \n",
       "L 67.956622 19.481189 \n",
       "L 68.374874 21.887053 \n",
       "L 68.790372 24.217907 \n",
       "L 69.203196 26.474711 \n",
       "L 69.613423 28.658328 \n",
       "L 70.021129 30.769653 \n",
       "L 70.426391 32.809526 \n",
       "L 70.829279 34.778746 \n",
       "L 71.229867 36.678109 \n",
       "L 71.628226 38.508382 \n",
       "L 72.024423 40.270285 \n",
       "L 72.418528 41.964535 \n",
       "L 72.810609 43.59182 \n",
       "L 73.200729 45.152795 \n",
       "L 73.588954 46.648101 \n",
       "L 73.975349 48.078349 \n",
       "L 74.359974 49.444128 \n",
       "L 74.742892 50.746013 \n",
       "L 75.124166 51.984554 \n",
       "L 75.503851 53.160272 \n",
       "L 75.88201 54.273678 \n",
       "L 76.258699 55.325259 \n",
       "L 76.633977 56.315483 \n",
       "L 77.007898 57.244795 \n",
       "L 77.380521 58.113622 \n",
       "L 77.751898 58.92238 \n",
       "L 78.122086 59.671458 \n",
       "L 78.491137 60.361227 \n",
       "L 78.859104 60.992046 \n",
       "L 79.226041 61.564253 \n",
       "L 79.591999 62.078168 \n",
       "L 79.95703 62.534097 \n",
       "L 80.321185 62.932329 \n",
       "L 80.684513 63.273133 \n",
       "L 81.047065 63.556768 \n",
       "L 81.408891 63.783469 \n",
       "L 81.770039 63.95346 \n",
       "L 82.130559 64.066952 \n",
       "L 82.490499 64.124136 \n",
       "L 82.849906 64.125188 \n",
       "L 83.208828 64.070272 \n",
       "L 83.567313 63.959533 \n",
       "L 83.925407 63.793106 \n",
       "L 84.283159 63.571107 \n",
       "L 84.640614 63.29364 \n",
       "L 84.997818 62.960792 \n",
       "L 85.354817 62.572639 \n",
       "L 85.711658 62.12924 \n",
       "L 86.068387 61.630641 \n",
       "L 86.425049 61.076872 \n",
       "L 86.781689 60.467951 \n",
       "L 87.138354 59.803882 \n",
       "L 87.495089 59.084653 \n",
       "L 87.851938 58.31024 \n",
       "L 88.208948 57.480604 \n",
       "L 88.566163 56.595689 \n",
       "L 88.923629 55.655434 \n",
       "L 89.281391 54.659752 \n",
       "L 89.639495 53.608552 \n",
       "L 89.997985 52.501725 \n",
       "L 90.356907 51.339145 \n",
       "L 90.716307 50.120677 \n",
       "L 91.076232 48.846169 \n",
       "L 91.436725 47.515452 \n",
       "L 91.797833 46.128352 \n",
       "L 92.159603 44.684673 \n",
       "L 92.52208 43.184203 \n",
       "L 92.885312 41.626718 \n",
       "L 93.249345 40.011984 \n",
       "L 93.614226 38.339747 \n",
       "L 93.980002 36.609738 \n",
       "L 94.346721 34.821669 \n",
       "L 94.714431 32.975251 \n",
       "L 95.083179 31.070167 \n",
       "L 95.453016 29.106081 \n",
       "L 95.823989 27.082656 \n",
       "L 96.196148 24.999528 \n",
       "L 96.569543 22.856319 \n",
       "L 96.944223 20.652641 \n",
       "L 97.32024 18.388082 \n",
       "L 97.697645 16.062204 \n",
       "L 98.076489 13.67458 \n",
       "L 98.456825 11.224743 \n",
       "L 98.838706 8.712201 \n",
       "L 99.222184 6.136477 \n",
       "L 99.607314 3.497055 \n",
       "L 99.994151 0.793378 \n",
       "L 100.245882 -1 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #800080; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 74.520244 -1 \n",
       "L 74.644258 -0.217352 \n",
       "L 75.052099 2.293359 \n",
       "L 75.457501 4.726037 \n",
       "L 75.860542 7.081697 \n",
       "L 76.261293 9.361256 \n",
       "L 76.659832 11.565665 \n",
       "L 77.05623 13.695816 \n",
       "L 77.450558 15.752556 \n",
       "L 77.842887 17.736732 \n",
       "L 78.233285 19.649157 \n",
       "L 78.621819 21.490593 \n",
       "L 79.008555 23.261816 \n",
       "L 79.393561 24.963534 \n",
       "L 79.776898 26.596449 \n",
       "L 80.15863 28.161242 \n",
       "L 80.538821 29.658564 \n",
       "L 80.91753 31.089045 \n",
       "L 81.294818 32.453287 \n",
       "L 81.670745 33.751867 \n",
       "L 82.045369 34.985356 \n",
       "L 82.418747 36.15428 \n",
       "L 82.790938 37.259155 \n",
       "L 83.161997 38.300488 \n",
       "L 83.531979 39.278746 \n",
       "L 83.90094 40.194388 \n",
       "L 84.268935 41.047838 \n",
       "L 84.636016 41.839525 \n",
       "L 85.002237 42.569836 \n",
       "L 85.36765 43.239153 \n",
       "L 85.732307 43.847837 \n",
       "L 86.09626 44.396223 \n",
       "L 86.45956 44.884642 \n",
       "L 86.822257 45.31339 \n",
       "L 87.184402 45.682762 \n",
       "L 87.546044 45.993029 \n",
       "L 87.907232 46.244446 \n",
       "L 88.268016 46.437247 \n",
       "L 88.628444 46.571657 \n",
       "L 88.988565 46.647877 \n",
       "L 89.348427 46.666103 \n",
       "L 89.708077 46.626498 \n",
       "L 90.067563 46.529226 \n",
       "L 90.426932 46.374428 \n",
       "L 90.786233 46.16223 \n",
       "L 91.145512 45.89274 \n",
       "L 91.504814 45.566056 \n",
       "L 91.864188 45.182257 \n",
       "L 92.223679 44.741409 \n",
       "L 92.583335 44.243563 \n",
       "L 92.943202 43.688754 \n",
       "L 93.303326 43.076996 \n",
       "L 93.663754 42.408305 \n",
       "L 94.024531 41.682664 \n",
       "L 94.385705 40.900053 \n",
       "L 94.747322 40.060431 \n",
       "L 95.109427 39.163745 \n",
       "L 95.472069 38.209929 \n",
       "L 95.835293 37.198898 \n",
       "L 96.199146 36.130551 \n",
       "L 96.563674 35.004783 \n",
       "L 96.928926 33.821457 \n",
       "L 97.294947 32.580441 \n",
       "L 97.661786 31.281574 \n",
       "L 98.029491 29.924678 \n",
       "L 98.398107 28.509579 \n",
       "L 98.767685 27.036058 \n",
       "L 99.138272 25.503911 \n",
       "L 99.509917 23.912898 \n",
       "L 99.882669 22.262779 \n",
       "L 100.256576 20.55328 \n",
       "L 100.63169 18.784121 \n",
       "L 101.00806 16.955016 \n",
       "L 101.385735 15.065644 \n",
       "L 101.764768 13.115682 \n",
       "L 102.145209 11.104785 \n",
       "L 102.52711 9.032594 \n",
       "L 102.910524 6.898731 \n",
       "L 103.295503 4.702794 \n",
       "L 103.682101 2.444373 \n",
       "L 104.070371 0.123056 \n",
       "L 104.254038 -1 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #800080; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 86.237398 -1 \n",
       "L 86.279712 -0.798433 \n",
       "L 86.65706 0.933394 \n",
       "L 87.033109 2.59377 \n",
       "L 87.407923 4.18341 \n",
       "L 87.781558 5.702961 \n",
       "L 88.154076 7.153093 \n",
       "L 88.525534 8.534447 \n",
       "L 88.89599 9.847594 \n",
       "L 89.2655 11.093146 \n",
       "L 89.634122 12.27164 \n",
       "L 90.001911 13.383606 \n",
       "L 90.368921 14.429579 \n",
       "L 90.735207 15.410032 \n",
       "L 91.100821 16.325447 \n",
       "L 91.465819 17.176258 \n",
       "L 91.830252 17.9629 \n",
       "L 92.194173 18.685775 \n",
       "L 92.557632 19.345276 \n",
       "L 92.920682 19.941778 \n",
       "L 93.283374 20.475618 \n",
       "L 93.645757 20.947132 \n",
       "L 94.007882 21.356632 \n",
       "L 94.369799 21.704405 \n",
       "L 94.731557 21.990733 \n",
       "L 95.093205 22.215872 \n",
       "L 95.454793 22.380062 \n",
       "L 95.816368 22.483527 \n",
       "L 96.177981 22.52647 \n",
       "L 96.539678 22.509075 \n",
       "L 96.901508 22.431517 \n",
       "L 97.26352 22.293951 \n",
       "L 97.62576 22.096516 \n",
       "L 97.988278 21.839332 \n",
       "L 98.351119 21.5225 \n",
       "L 98.714332 21.146112 \n",
       "L 99.077966 20.710244 \n",
       "L 99.442066 20.21495 \n",
       "L 99.806681 19.660272 \n",
       "L 100.171859 19.046228 \n",
       "L 100.537647 18.372834 \n",
       "L 100.904093 17.640083 \n",
       "L 101.271244 16.847953 \n",
       "L 101.639149 15.996405 \n",
       "L 102.007856 15.085385 \n",
       "L 102.377413 14.114817 \n",
       "L 102.747868 13.084622 \n",
       "L 103.119271 11.9947 \n",
       "L 103.491669 10.844926 \n",
       "L 103.865112 9.635181 \n",
       "L 104.239648 8.36531 \n",
       "L 104.61533 7.035142 \n",
       "L 104.992204 5.644503 \n",
       "L 105.370323 4.19319 \n",
       "L 105.749736 2.681002 \n",
       "L 106.130494 1.1077 \n",
       "L 106.512649 -0.526946 \n",
       "L 106.619628 -1 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #800080; stroke-width: 1.5\"/>\n",
       "    <path clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #800080; stroke-width: 1.5\"/>\n",
       "    <path clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #800080; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 29.001712 -1 \n",
       "L 29.636011 0.591962 \n",
       "L 30.337515 2.295782 \n",
       "L 31.035915 3.935097 \n",
       "L 31.731315 5.510267 \n",
       "L 32.42381 7.021612 \n",
       "L 33.113501 8.469429 \n",
       "L 33.800488 9.854036 \n",
       "L 34.484863 11.17568 \n",
       "L 35.166727 12.434637 \n",
       "L 35.846175 13.631139 \n",
       "L 36.523299 14.765395 \n",
       "L 37.198194 15.837629 \n",
       "L 37.870955 16.848027 \n",
       "L 38.541671 17.796748 \n",
       "L 39.210436 18.683938 \n",
       "L 39.877343 19.509759 \n",
       "L 40.542479 20.274313 \n",
       "L 41.205937 20.977709 \n",
       "L 41.867807 21.620034 \n",
       "L 42.528176 22.201351 \n",
       "L 43.187135 22.721721 \n",
       "L 43.844773 23.181181 \n",
       "L 44.501178 23.579751 \n",
       "L 45.156437 23.917436 \n",
       "L 45.81064 24.194222 \n",
       "L 46.463873 24.410081 \n",
       "L 47.116226 24.56496 \n",
       "L 47.767786 24.658804 \n",
       "L 48.418638 24.691534 \n",
       "L 49.068873 24.663046 \n",
       "L 49.718576 24.573241 \n",
       "L 50.367837 24.421976 \n",
       "L 51.016742 24.20911 \n",
       "L 51.66538 23.934479 \n",
       "L 52.313839 23.597901 \n",
       "L 52.962206 23.19917 \n",
       "L 53.610572 22.738076 \n",
       "L 54.259024 22.214384 \n",
       "L 54.907653 21.627847 \n",
       "L 55.556547 20.978174 \n",
       "L 56.205797 20.265096 \n",
       "L 56.855494 19.48829 \n",
       "L 57.505729 18.647435 \n",
       "L 58.156593 17.742192 \n",
       "L 58.808181 16.772175 \n",
       "L 59.460583 15.737009 \n",
       "L 60.113893 14.636296 \n",
       "L 60.768207 13.469603 \n",
       "L 61.423619 12.236469 \n",
       "L 62.080227 10.936444 \n",
       "L 62.738127 9.569027 \n",
       "L 63.397417 8.133724 \n",
       "L 64.058197 6.629969 \n",
       "L 64.720566 5.057236 \n",
       "L 65.384627 3.414921 \n",
       "L 66.05048 1.702437 \n",
       "L 66.718231 -0.080855 \n",
       "L 67.05014 -1 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #800080; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 30.591837 14.057303 \n",
       "L 31.289385 16.075291 \n",
       "L 31.983578 18.029407 \n",
       "L 32.674515 19.920076 \n",
       "L 33.362296 21.747719 \n",
       "L 34.047023 23.512719 \n",
       "L 34.72879 25.215461 \n",
       "L 35.407694 26.856287 \n",
       "L 36.083834 28.435554 \n",
       "L 36.757303 29.95358 \n",
       "L 37.428194 31.410664 \n",
       "L 38.0966 32.807101 \n",
       "L 38.762615 34.143165 \n",
       "L 39.426327 35.419109 \n",
       "L 40.087828 36.635177 \n",
       "L 40.747211 37.791589 \n",
       "L 41.404559 38.888551 \n",
       "L 42.059964 39.926255 \n",
       "L 42.713516 40.904886 \n",
       "L 43.365297 41.824594 \n",
       "L 44.015397 42.685522 \n",
       "L 44.663904 43.487811 \n",
       "L 45.310899 44.231557 \n",
       "L 45.95647 44.916868 \n",
       "L 46.600705 45.543831 \n",
       "L 47.243684 46.112504 \n",
       "L 47.885493 46.622943 \n",
       "L 48.526217 47.075185 \n",
       "L 49.165941 47.469244 \n",
       "L 49.804745 47.805134 \n",
       "L 50.442717 48.08284 \n",
       "L 51.079937 48.302339 \n",
       "L 51.716492 48.463589 \n",
       "L 52.352462 48.566534 \n",
       "L 52.987932 48.611104 \n",
       "L 53.622986 48.597208 \n",
       "L 54.257707 48.524746 \n",
       "L 54.892178 48.393597 \n",
       "L 55.526485 48.203627 \n",
       "L 56.160709 47.954687 \n",
       "L 56.794936 47.646609 \n",
       "L 57.42925 47.279204 \n",
       "L 58.063736 46.852281 \n",
       "L 58.698479 46.365624 \n",
       "L 59.333565 45.818985 \n",
       "L 59.969079 45.212132 \n",
       "L 60.605108 44.544789 \n",
       "L 61.241739 43.816674 \n",
       "L 61.879059 43.027482 \n",
       "L 62.517157 42.176897 \n",
       "L 63.156122 41.264581 \n",
       "L 63.79604 40.290171 \n",
       "L 64.437004 39.253299 \n",
       "L 65.079104 38.153575 \n",
       "L 65.722432 36.990579 \n",
       "L 66.36708 35.763888 \n",
       "L 67.013141 34.473039 \n",
       "L 67.66071 33.117558 \n",
       "L 68.309883 31.69696 \n",
       "L 68.960754 30.21073 \n",
       "L 69.613423 28.658328 \n",
       "L 70.267987 27.039214 \n",
       "L 70.924547 25.352786 \n",
       "L 71.583204 23.59846 \n",
       "L 72.24406 21.775592 \n",
       "L 72.907218 19.883553 \n",
       "L 73.572785 17.921663 \n",
       "L 74.240866 15.889226 \n",
       "L 74.911572 13.785518 \n",
       "L 75.58501 11.609778 \n",
       "L 76.261293 9.361256 \n",
       "L 76.940536 7.039124 \n",
       "L 77.622852 4.642578 \n",
       "L 78.30836 2.170726 \n",
       "L 78.997178 -0.377287 \n",
       "L 79.161389 -1 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #800080; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 35.360539 31.975464 \n",
       "L 36.037999 33.931585 \n",
       "L 36.712422 35.826169 \n",
       "L 37.383899 37.659601 \n",
       "L 38.052527 39.432282 \n",
       "L 38.718402 41.14459 \n",
       "L 39.381612 42.796867 \n",
       "L 40.042251 44.389456 \n",
       "L 40.700409 45.92268 \n",
       "L 41.356178 47.39685 \n",
       "L 42.009643 48.812246 \n",
       "L 42.660894 50.16915 \n",
       "L 43.31002 51.467816 \n",
       "L 43.957105 52.708488 \n",
       "L 44.602235 53.891391 \n",
       "L 45.245499 55.016742 \n",
       "L 45.886978 56.084735 \n",
       "L 46.526757 57.095552 \n",
       "L 47.164922 58.049362 \n",
       "L 47.801553 58.946314 \n",
       "L 48.436734 59.786547 \n",
       "L 49.070549 60.570188 \n",
       "L 49.703078 61.297339 \n",
       "L 50.334403 61.968092 \n",
       "L 50.964607 62.582532 \n",
       "L 51.593768 63.14072 \n",
       "L 52.221969 63.642708 \n",
       "L 52.849292 64.088529 \n",
       "L 53.475815 64.478205 \n",
       "L 54.101618 64.811743 \n",
       "L 54.726784 65.089134 \n",
       "L 55.351391 65.310356 \n",
       "L 55.97552 65.47537 \n",
       "L 56.599252 65.584126 \n",
       "L 57.222665 65.636556 \n",
       "L 57.845841 65.632582 \n",
       "L 58.46886 65.572104 \n",
       "L 59.091804 65.455014 \n",
       "L 59.714752 65.281183 \n",
       "L 60.337785 65.050474 \n",
       "L 60.960984 64.762727 \n",
       "L 61.584432 64.417772 \n",
       "L 62.20821 64.015421 \n",
       "L 62.832401 63.555474 \n",
       "L 63.457086 63.037709 \n",
       "L 64.082349 62.461894 \n",
       "L 64.708275 61.82778 \n",
       "L 65.334945 61.135096 \n",
       "L 65.962446 60.383561 \n",
       "L 66.590862 59.572876 \n",
       "L 67.220281 58.702724 \n",
       "L 67.850786 57.772766 \n",
       "L 68.482466 56.782657 \n",
       "L 69.11541 55.732028 \n",
       "L 69.749705 54.620483 \n",
       "L 70.385442 53.447631 \n",
       "L 71.022711 52.213033 \n",
       "L 71.661603 50.916258 \n",
       "L 72.302213 49.55683 \n",
       "L 72.944631 48.134282 \n",
       "L 73.588954 46.648101 \n",
       "L 74.235277 45.097773 \n",
       "L 74.883697 43.48275 \n",
       "L 75.534313 41.80247 \n",
       "L 76.187224 40.056342 \n",
       "L 76.842529 38.243767 \n",
       "L 77.500334 36.364108 \n",
       "L 78.160739 34.416711 \n",
       "L 78.823853 32.400916 \n",
       "L 79.489781 30.316001 \n",
       "L 80.15863 28.161242 \n",
       "L 80.830514 25.935886 \n",
       "L 81.505542 23.639169 \n",
       "L 82.18383 21.270275 \n",
       "L 82.865493 18.828379 \n",
       "L 83.550649 16.312627 \n",
       "L 84.239417 13.722126 \n",
       "L 84.931924 11.055946 \n",
       "L 85.628288 8.313158 \n",
       "L 86.328639 5.492778 \n",
       "L 87.033109 2.59377 \n",
       "L 87.741825 -0.384859 \n",
       "L 87.885207 -1 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #800080; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 39.754279 43.452644 \n",
       "L 40.41568 45.363481 \n",
       "L 41.074314 47.214443 \n",
       "L 41.730266 49.005908 \n",
       "L 42.38363 50.738254 \n",
       "L 43.034496 52.411846 \n",
       "L 43.68295 54.027011 \n",
       "L 44.32908 55.584078 \n",
       "L 44.972973 57.083357 \n",
       "L 45.614717 58.525143 \n",
       "L 46.254393 59.90971 \n",
       "L 46.892088 61.237322 \n",
       "L 47.527888 62.508232 \n",
       "L 48.161871 63.722665 \n",
       "L 48.794122 64.880842 \n",
       "L 49.424726 65.982973 \n",
       "L 50.053759 67.029238 \n",
       "L 50.681305 68.019814 \n",
       "L 51.307446 68.954868 \n",
       "L 51.932258 69.834539 \n",
       "L 52.555824 70.658962 \n",
       "L 53.178224 71.428258 \n",
       "L 53.799534 72.142525 \n",
       "L 54.419834 72.801858 \n",
       "L 55.039205 73.406333 \n",
       "L 55.657721 73.956009 \n",
       "L 56.275463 74.450938 \n",
       "L 56.892509 74.891152 \n",
       "L 57.508937 75.276674 \n",
       "L 58.124822 75.60751 \n",
       "L 58.740246 75.883653 \n",
       "L 59.355283 76.105082 \n",
       "L 59.970013 76.271761 \n",
       "L 60.584514 76.383643 \n",
       "L 61.198861 76.440664 \n",
       "L 61.813136 76.442749 \n",
       "L 62.427414 76.389804 \n",
       "L 63.041775 76.281726 \n",
       "L 63.656297 76.118392 \n",
       "L 64.271059 75.899672 \n",
       "L 64.886139 75.625414 \n",
       "L 65.501618 75.295457 \n",
       "L 66.117574 74.909622 \n",
       "L 66.734089 74.467716 \n",
       "L 67.351243 73.96953 \n",
       "L 67.969116 73.414843 \n",
       "L 68.58779 72.803414 \n",
       "L 69.207348 72.134991 \n",
       "L 69.827871 71.409303 \n",
       "L 70.449444 70.626062 \n",
       "L 71.072151 69.784969 \n",
       "L 71.696074 68.885701 \n",
       "L 72.3213 67.927932 \n",
       "L 72.947914 66.911304 \n",
       "L 73.576005 65.835444 \n",
       "L 74.205659 64.699976 \n",
       "L 74.836966 63.504487 \n",
       "L 75.470014 62.248556 \n",
       "L 76.104895 60.931744 \n",
       "L 76.741699 59.553595 \n",
       "L 77.380521 58.113622 \n",
       "L 78.021453 56.611339 \n",
       "L 78.664591 55.04622 \n",
       "L 79.310031 53.417733 \n",
       "L 79.957871 51.725313 \n",
       "L 80.608208 49.96839 \n",
       "L 81.261145 48.146363 \n",
       "L 81.916782 46.258598 \n",
       "L 82.575224 44.30447 \n",
       "L 83.236575 42.283292 \n",
       "L 83.90094 40.194388 \n",
       "L 84.568431 38.037022 \n",
       "L 85.239153 35.81048 \n",
       "L 85.913223 33.513981 \n",
       "L 86.590752 31.14674 \n",
       "L 87.271855 28.707948 \n",
       "L 87.956651 26.196754 \n",
       "L 88.645263 23.612274 \n",
       "L 89.337808 20.953625 \n",
       "L 90.034413 18.219873 \n",
       "L 90.735207 15.410032 \n",
       "L 91.440314 12.523161 \n",
       "L 92.149871 9.558207 \n",
       "L 92.864012 6.514102 \n",
       "L 93.582871 3.389794 \n",
       "L 94.306591 0.184147 \n",
       "L 94.569023 -1 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #800080; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 43.848465 48.975591 \n",
       "L 44.497355 50.855839 \n",
       "L 45.143708 52.6773 \n",
       "L 45.787606 54.440342 \n",
       "L 46.42914 56.145333 \n",
       "L 47.068397 57.792623 \n",
       "L 47.705459 59.38253 \n",
       "L 48.340411 60.915377 \n",
       "L 48.973338 62.391461 \n",
       "L 49.604324 63.811073 \n",
       "L 50.233449 65.174473 \n",
       "L 50.860795 66.481921 \n",
       "L 51.486445 67.733661 \n",
       "L 52.110478 68.929913 \n",
       "L 52.732973 70.070891 \n",
       "L 53.354013 71.156797 \n",
       "L 53.973673 72.187809 \n",
       "L 54.592033 73.164099 \n",
       "L 55.209174 74.085826 \n",
       "L 55.825169 74.953128 \n",
       "L 56.440098 75.766135 \n",
       "L 57.05404 76.524964 \n",
       "L 57.667069 77.229713 \n",
       "L 58.279262 77.880472 \n",
       "L 58.890699 78.477316 \n",
       "L 59.501451 79.020303 \n",
       "L 60.111597 79.509482 \n",
       "L 60.721214 79.944886 \n",
       "L 61.330377 80.326538 \n",
       "L 61.939161 80.654442 \n",
       "L 62.547644 80.928594 \n",
       "L 63.1559 81.148973 \n",
       "L 63.764007 81.315543 \n",
       "L 64.37204 81.428261 \n",
       "L 64.980074 81.487066 \n",
       "L 65.588188 81.491882 \n",
       "L 66.196456 81.442621 \n",
       "L 66.804957 81.339181 \n",
       "L 67.413767 81.181448 \n",
       "L 68.022963 80.969291 \n",
       "L 68.632624 80.702566 \n",
       "L 69.242826 80.381116 \n",
       "L 69.853648 80.004769 \n",
       "L 70.46517 79.573337 \n",
       "L 71.07747 79.086618 \n",
       "L 71.690628 78.544399 \n",
       "L 72.304724 77.946446 \n",
       "L 72.919839 77.292517 \n",
       "L 73.536054 76.582346 \n",
       "L 74.153452 75.815658 \n",
       "L 74.772116 74.992162 \n",
       "L 75.392127 74.111546 \n",
       "L 76.01357 73.173494 \n",
       "L 76.636531 72.177659 \n",
       "L 77.261096 71.123682 \n",
       "L 77.88735 70.011198 \n",
       "L 78.515381 68.839807 \n",
       "L 79.145278 67.609105 \n",
       "L 79.777131 66.318661 \n",
       "L 80.411029 64.96804 \n",
       "L 81.047065 63.556768 \n",
       "L 81.685332 62.084375 \n",
       "L 82.325923 60.550353 \n",
       "L 82.968933 58.954189 \n",
       "L 83.61446 57.295333 \n",
       "L 84.262601 55.573237 \n",
       "L 84.913455 53.787319 \n",
       "L 85.567123 51.936967 \n",
       "L 86.223708 50.021576 \n",
       "L 86.883313 48.040482 \n",
       "L 87.546044 45.993029 \n",
       "L 88.212008 43.878516 \n",
       "L 88.881314 41.696242 \n",
       "L 89.554073 39.445458 \n",
       "L 90.230398 37.125405 \n",
       "L 90.910403 34.735299 \n",
       "L 91.594204 32.274323 \n",
       "L 92.281924 29.741622 \n",
       "L 92.973679 27.136344 \n",
       "L 93.669594 24.457587 \n",
       "L 94.369799 21.704405 \n",
       "L 95.074415 18.875874 \n",
       "L 95.783578 15.970994 \n",
       "L 96.497421 12.988731 \n",
       "L 97.216077 9.928075 \n",
       "L 97.939688 6.787914 \n",
       "L 98.668397 3.567125 \n",
       "L 99.402344 0.264578 \n",
       "L 99.678507 -1 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #800080; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 47.706352 48.851906 \n",
       "L 48.345925 50.715022 \n",
       "L 48.983159 52.519914 \n",
       "L 49.618136 54.266942 \n",
       "L 50.250943 55.956467 \n",
       "L 50.881666 57.588833 \n",
       "L 51.510383 59.164353 \n",
       "L 52.137179 60.683343 \n",
       "L 52.762136 62.146095 \n",
       "L 53.385335 63.552893 \n",
       "L 54.006855 64.903995 \n",
       "L 54.626776 66.199655 \n",
       "L 55.245179 67.440113 \n",
       "L 55.862139 68.625585 \n",
       "L 56.477737 69.756281 \n",
       "L 57.092052 70.832399 \n",
       "L 57.705158 71.854113 \n",
       "L 58.317133 72.821594 \n",
       "L 58.928056 73.734998 \n",
       "L 59.538 74.594458 \n",
       "L 60.147042 75.400104 \n",
       "L 60.755259 76.15205 \n",
       "L 61.362725 76.850393 \n",
       "L 61.969515 77.495219 \n",
       "L 62.575708 78.086605 \n",
       "L 63.181373 78.624606 \n",
       "L 63.786589 79.10927 \n",
       "L 64.39143 79.540631 \n",
       "L 64.995972 79.918709 \n",
       "L 65.600287 80.243511 \n",
       "L 66.204452 80.51503 \n",
       "L 66.808541 80.733247 \n",
       "L 67.41263 80.898126 \n",
       "L 68.016793 81.009626 \n",
       "L 68.621105 81.067684 \n",
       "L 69.225644 81.072229 \n",
       "L 69.830482 81.023173 \n",
       "L 70.435698 80.920417 \n",
       "L 71.041367 80.763845 \n",
       "L 71.647565 80.553333 \n",
       "L 72.254369 80.288737 \n",
       "L 72.861856 79.969903 \n",
       "L 73.470105 79.596662 \n",
       "L 74.079193 79.168832 \n",
       "L 74.689199 78.686211 \n",
       "L 75.300201 78.148592 \n",
       "L 75.91228 77.555743 \n",
       "L 76.525515 76.907429 \n",
       "L 77.139988 76.203388 \n",
       "L 77.75578 75.443349 \n",
       "L 78.372974 74.627027 \n",
       "L 78.991651 73.754115 \n",
       "L 79.611895 72.824304 \n",
       "L 80.233791 71.837253 \n",
       "L 80.857425 70.792609 \n",
       "L 81.482883 69.690015 \n",
       "L 82.110252 68.529076 \n",
       "L 82.739619 67.309398 \n",
       "L 83.371075 66.030558 \n",
       "L 84.00471 64.692126 \n",
       "L 84.640614 63.29364 \n",
       "L 85.278881 61.83464 \n",
       "L 85.919604 60.314623 \n",
       "L 86.562878 58.733089 \n",
       "L 87.208801 57.089499 \n",
       "L 87.857468 55.383312 \n",
       "L 88.508981 53.613957 \n",
       "L 89.163438 51.780842 \n",
       "L 89.820943 49.883363 \n",
       "L 90.481599 47.920879 \n",
       "L 91.145512 45.89274 \n",
       "L 91.812789 43.798259 \n",
       "L 92.483538 41.636752 \n",
       "L 93.157871 39.407482 \n",
       "L 93.835901 37.109707 \n",
       "L 94.517741 34.742659 \n",
       "L 95.203509 32.305533 \n",
       "L 95.893326 29.7975 \n",
       "L 96.587309 27.217717 \n",
       "L 97.285583 24.565302 \n",
       "L 97.988278 21.839332 \n",
       "L 98.695515 19.038899 \n",
       "L 99.40743 16.163026 \n",
       "L 100.124157 13.210698 \n",
       "L 100.845828 10.180923 \n",
       "L 101.572586 7.072623 \n",
       "L 102.304573 3.884691 \n",
       "L 103.041931 0.616026 \n",
       "L 103.400231 -1 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #800080; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 51.382469 43.234717 \n",
       "L 52.015666 45.093447 \n",
       "L 52.6467 46.894017 \n",
       "L 53.27565 48.636783 \n",
       "L 53.902601 50.322104 \n",
       "L 54.527637 51.950324 \n",
       "L 55.150836 53.521751 \n",
       "L 55.772281 55.036701 \n",
       "L 56.392052 56.495462 \n",
       "L 57.010231 57.898319 \n",
       "L 57.626892 59.245525 \n",
       "L 58.242116 60.537334 \n",
       "L 58.855983 61.773983 \n",
       "L 59.468567 62.955687 \n",
       "L 60.079946 64.082654 \n",
       "L 60.6902 65.155082 \n",
       "L 61.299401 66.173144 \n",
       "L 61.907626 67.137007 \n",
       "L 62.514953 68.046827 \n",
       "L 63.121455 68.902737 \n",
       "L 63.727207 69.704864 \n",
       "L 64.332287 70.453323 \n",
       "L 64.936765 71.148208 \n",
       "L 65.540719 71.789606 \n",
       "L 66.144224 72.377592 \n",
       "L 66.747351 72.912221 \n",
       "L 67.350177 73.39354 \n",
       "L 67.952776 73.821582 \n",
       "L 68.555221 74.196368 \n",
       "L 69.157587 74.517902 \n",
       "L 69.75995 74.78618 \n",
       "L 70.362381 75.00118 \n",
       "L 70.964958 75.162868 \n",
       "L 71.567754 75.2712 \n",
       "L 72.170844 75.326116 \n",
       "L 72.774304 75.327543 \n",
       "L 73.378207 75.275393 \n",
       "L 73.982632 75.169568 \n",
       "L 74.587653 75.009953 \n",
       "L 75.193347 74.796422 \n",
       "L 75.79979 74.528834 \n",
       "L 76.407059 74.207034 \n",
       "L 77.015233 73.830854 \n",
       "L 77.624389 73.400111 \n",
       "L 78.234605 72.914606 \n",
       "L 78.845961 72.374131 \n",
       "L 79.458536 71.778457 \n",
       "L 80.072411 71.127347 \n",
       "L 80.687667 70.420543 \n",
       "L 81.304385 69.657774 \n",
       "L 81.922649 68.838755 \n",
       "L 82.542539 67.963181 \n",
       "L 83.164141 67.030746 \n",
       "L 83.787539 66.041107 \n",
       "L 84.41282 64.993917 \n",
       "L 85.040069 63.888817 \n",
       "L 85.669374 62.725417 \n",
       "L 86.300825 61.503321 \n",
       "L 86.93451 60.222111 \n",
       "L 87.57052 58.881359 \n",
       "L 88.208948 57.480604 \n",
       "L 88.849886 56.019387 \n",
       "L 89.493428 54.497211 \n",
       "L 90.139672 52.913577 \n",
       "L 90.788713 51.267948 \n",
       "L 91.440649 49.559788 \n",
       "L 92.095582 47.788531 \n",
       "L 92.753611 45.953579 \n",
       "L 93.414841 44.054341 \n",
       "L 94.079376 42.090172 \n",
       "L 94.747322 40.060431 \n",
       "L 95.418788 37.964433 \n",
       "L 96.093882 35.801498 \n",
       "L 96.772718 33.570894 \n",
       "L 97.45541 31.271882 \n",
       "L 98.142071 28.903697 \n",
       "L 98.832821 26.465546 \n",
       "L 99.527782 23.956594 \n",
       "L 100.227072 21.376013 \n",
       "L 100.930818 18.722923 \n",
       "L 101.639149 15.996405 \n",
       "L 102.352191 13.195558 \n",
       "L 103.070078 10.319411 \n",
       "L 103.792948 7.366956 \n",
       "L 104.520934 4.337214 \n",
       "L 105.254178 1.2291 \n",
       "L 105.770721 -1 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #800080; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 54.925284 32.136092 \n",
       "L 55.554884 34.002924 \n",
       "L 56.182475 35.81117 \n",
       "L 56.808135 37.561189 \n",
       "L 57.431949 39.253341 \n",
       "L 58.054001 40.887971 \n",
       "L 58.674366 42.465392 \n",
       "L 59.293128 43.985916 \n",
       "L 59.910366 45.449836 \n",
       "L 60.52616 46.857434 \n",
       "L 61.140585 48.208969 \n",
       "L 61.753722 49.504693 \n",
       "L 62.365649 50.744845 \n",
       "L 62.97644 51.92964 \n",
       "L 63.586173 53.059288 \n",
       "L 64.194926 54.133986 \n",
       "L 64.802772 55.153908 \n",
       "L 65.409788 56.11922 \n",
       "L 66.016052 57.030081 \n",
       "L 66.621634 57.886623 \n",
       "L 67.226612 58.688973 \n",
       "L 67.831061 59.437247 \n",
       "L 68.435054 60.131537 \n",
       "L 69.038666 60.771932 \n",
       "L 69.641974 61.358504 \n",
       "L 70.245048 61.89131 \n",
       "L 70.847964 62.370397 \n",
       "L 71.450797 62.795796 \n",
       "L 72.053622 63.167527 \n",
       "L 72.656511 63.485595 \n",
       "L 73.259541 63.749994 \n",
       "L 73.862784 63.960702 \n",
       "L 74.466318 64.117682 \n",
       "L 75.070215 64.220892 \n",
       "L 75.674551 64.270268 \n",
       "L 76.279404 64.265739 \n",
       "L 76.884845 64.207213 \n",
       "L 77.490955 64.09459 \n",
       "L 78.097807 63.927756 \n",
       "L 78.705479 63.706582 \n",
       "L 79.314047 63.430924 \n",
       "L 79.923591 63.100628 \n",
       "L 80.534187 62.715523 \n",
       "L 81.145915 62.275423 \n",
       "L 81.758853 61.780127 \n",
       "L 82.373082 61.229426 \n",
       "L 82.988682 60.623088 \n",
       "L 83.605733 59.960872 \n",
       "L 84.224318 59.24252 \n",
       "L 84.844519 58.467756 \n",
       "L 85.46642 57.636295 \n",
       "L 86.090104 56.747828 \n",
       "L 86.715656 55.802043 \n",
       "L 87.343162 54.7986 \n",
       "L 87.972709 53.737142 \n",
       "L 88.604384 52.617311 \n",
       "L 89.238277 51.438712 \n",
       "L 89.874476 50.200945 \n",
       "L 90.513075 48.90359 \n",
       "L 91.154162 47.546214 \n",
       "L 91.797833 46.128352 \n",
       "L 92.444182 44.649541 \n",
       "L 93.093304 43.109282 \n",
       "L 93.745298 41.507065 \n",
       "L 94.400261 39.842356 \n",
       "L 95.058293 38.11461 \n",
       "L 95.719496 36.323257 \n",
       "L 96.383973 34.467695 \n",
       "L 97.05183 32.547327 \n",
       "L 97.723172 30.561502 \n",
       "L 98.398107 28.509579 \n",
       "L 99.076747 26.390857 \n",
       "L 99.759201 24.204658 \n",
       "L 100.445586 21.950242 \n",
       "L 101.136017 19.626864 \n",
       "L 101.83061 17.233757 \n",
       "L 102.529487 14.770113 \n",
       "L 103.232772 12.235095 \n",
       "L 103.940587 9.627867 \n",
       "L 104.653059 6.947543 \n",
       "L 105.370323 4.19319 \n",
       "L 106.092504 1.363919 \n",
       "L 106.684249 -1 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #800080; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 58.379374 15.431044 \n",
       "L 59.008063 17.318642 \n",
       "L 59.634881 19.146737 \n",
       "L 60.259907 20.915683 \n",
       "L 60.883225 22.62585 \n",
       "L 61.504919 24.277591 \n",
       "L 62.125064 25.871219 \n",
       "L 62.743743 27.407053 \n",
       "L 63.361036 28.885392 \n",
       "L 63.977024 30.306523 \n",
       "L 64.591781 31.670705 \n",
       "L 65.205387 32.978199 \n",
       "L 65.817921 34.22924 \n",
       "L 66.429458 35.424053 \n",
       "L 67.040075 36.562847 \n",
       "L 67.649851 37.645822 \n",
       "L 68.258859 38.673156 \n",
       "L 68.867177 39.645019 \n",
       "L 69.474881 40.561567 \n",
       "L 70.082045 41.422935 \n",
       "L 70.688744 42.229253 \n",
       "L 71.295056 42.980638 \n",
       "L 71.901052 43.677183 \n",
       "L 72.506809 44.318973 \n",
       "L 73.112404 44.906086 \n",
       "L 73.717909 45.438577 \n",
       "L 74.323398 45.916493 \n",
       "L 74.92895 46.339863 \n",
       "L 75.534637 46.708709 \n",
       "L 76.140534 47.023033 \n",
       "L 76.746718 47.282828 \n",
       "L 77.353263 47.488071 \n",
       "L 77.960245 47.638724 \n",
       "L 78.567741 47.73474 \n",
       "L 79.175825 47.776056 \n",
       "L 79.784574 47.762595 \n",
       "L 80.394066 47.694267 \n",
       "L 81.004377 47.570966 \n",
       "L 81.615585 47.392574 \n",
       "L 82.227767 47.15896 \n",
       "L 82.841001 46.869977 \n",
       "L 83.455368 46.525463 \n",
       "L 84.070945 46.125244 \n",
       "L 84.687814 45.669131 \n",
       "L 85.306054 45.156918 \n",
       "L 85.925747 44.588388 \n",
       "L 86.546974 43.963308 \n",
       "L 87.169818 43.281425 \n",
       "L 87.794363 42.542477 \n",
       "L 88.420691 41.746186 \n",
       "L 89.04889 40.892256 \n",
       "L 89.679043 39.98037 \n",
       "L 90.311237 39.010209 \n",
       "L 90.945561 37.981427 \n",
       "L 91.582102 36.893657 \n",
       "L 92.22095 35.746535 \n",
       "L 92.862196 34.539653 \n",
       "L 93.505933 33.272609 \n",
       "L 94.152252 31.944962 \n",
       "L 94.801248 30.556279 \n",
       "L 95.453016 29.106081 \n",
       "L 96.107654 27.593895 \n",
       "L 96.76526 26.019209 \n",
       "L 97.425932 24.381504 \n",
       "L 98.089774 22.680231 \n",
       "L 98.756884 20.914835 \n",
       "L 99.427371 19.084727 \n",
       "L 100.101338 17.189302 \n",
       "L 100.778894 15.227949 \n",
       "L 101.460148 13.2 \n",
       "L 102.145209 11.104785 \n",
       "L 102.834194 8.941609 \n",
       "L 103.527215 6.709767 \n",
       "L 104.224392 4.408505 \n",
       "L 104.925842 2.037062 \n",
       "L 105.631686 -0.405348 \n",
       "L 105.799699 -1 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #800080; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 63.876458 -1 \n",
       "L 64.298929 0.17525 \n",
       "L 64.922882 1.855301 \n",
       "L 65.545416 3.475841 \n",
       "L 66.166612 5.037174 \n",
       "L 66.786551 6.539622 \n",
       "L 67.405315 7.983472 \n",
       "L 68.022978 9.368992 \n",
       "L 68.639623 10.696448 \n",
       "L 69.255328 11.966085 \n",
       "L 69.870168 13.17813 \n",
       "L 70.484223 14.332805 \n",
       "L 71.097571 15.430305 \n",
       "L 71.710287 16.470816 \n",
       "L 72.322449 17.45451 \n",
       "L 72.934136 18.381553 \n",
       "L 73.54542 19.25208 \n",
       "L 74.156379 20.066218 \n",
       "L 74.767092 20.824093 \n",
       "L 75.37763 21.525791 \n",
       "L 75.988073 22.171409 \n",
       "L 76.598498 22.761022 \n",
       "L 77.208976 23.294683 \n",
       "L 77.819587 23.772442 \n",
       "L 78.430408 24.194328 \n",
       "L 79.041513 24.560354 \n",
       "L 79.652978 24.870531 \n",
       "L 80.264882 25.124845 \n",
       "L 80.8773 25.323271 \n",
       "L 81.49031 25.465771 \n",
       "L 82.10399 25.552294 \n",
       "L 82.718415 25.582772 \n",
       "L 83.333666 25.557124 \n",
       "L 83.94982 25.475256 \n",
       "L 84.566956 25.337057 \n",
       "L 85.185153 25.142405 \n",
       "L 85.804491 24.891163 \n",
       "L 86.425049 24.583178 \n",
       "L 87.04691 24.218277 \n",
       "L 87.670153 23.796286 \n",
       "L 88.294862 23.317007 \n",
       "L 88.921117 22.780215 \n",
       "L 89.549004 22.185699 \n",
       "L 90.178605 21.533207 \n",
       "L 90.810006 20.822483 \n",
       "L 91.443291 20.053251 \n",
       "L 92.078549 19.225222 \n",
       "L 92.715866 18.33809 \n",
       "L 93.355329 17.391526 \n",
       "L 93.997028 16.385196 \n",
       "L 94.641054 15.318745 \n",
       "L 95.287497 14.191796 \n",
       "L 95.93645 13.003965 \n",
       "L 96.588007 11.75483 \n",
       "L 97.242262 10.443966 \n",
       "L 97.899312 9.070933 \n",
       "L 98.559253 7.635266 \n",
       "L 99.222184 6.136477 \n",
       "L 99.888205 4.57408 \n",
       "L 100.557419 2.94753 \n",
       "L 101.229927 1.256303 \n",
       "L 101.905835 -0.500183 \n",
       "L 102.092182 -1 \n",
       "\" clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #800080; stroke-width: 1.5\"/>\n",
       "    <path clip-path=\"url(#p9d2841bd97)\" style=\"fill: none; stroke: #800080; stroke-width: 1.5\"/>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"p9d2841bd97\">\n",
       "   <rect x=\"7.2\" y=\"7.2\" width=\"138.6\" height=\"138.6\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<Figure size 350x250 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Construct grid and compute function\n",
    "x, y = torch.meshgrid(torch.linspace(-2, 2, 101),\n",
    "                   torch.linspace(-2, 2, 101))\n",
    "\n",
    "z = x*torch.exp(- x**2 - y**2)\n",
    "\n",
    "# Compute approximating quadratic with gradient and Hessian at (1, 0)\n",
    "w = torch.exp(torch.tensor([-1.]))*(-1 - (x + 1) + 2 * (x + 1)**2 + 2 * y**2)\n",
    "\n",
    "# Plot function\n",
    "ax = d2l.plt.figure().add_subplot(111, projection='3d')\n",
    "ax.plot_wireframe(x.numpy(), y.numpy(), z.numpy(),\n",
    "                  **{'rstride': 10, 'cstride': 10})\n",
    "ax.plot_wireframe(x.numpy(), y.numpy(), w.numpy(),\n",
    "                  **{'rstride': 10, 'cstride': 10}, color='purple')\n",
    "d2l.plt.xlabel('x')\n",
    "d2l.plt.ylabel('y')\n",
    "d2l.set_figsize()\n",
    "ax.set_xlim(-2, 2)\n",
    "ax.set_ylim(-2, 2)\n",
    "ax.set_zlim(-1, 1)\n",
    "ax.dist = 12"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2655e0f4",
   "metadata": {
    "origin_pos": 20
   },
   "source": [
    "This forms the basis for Newton's Algorithm discussed in :numref:`sec_gd`, where we perform numerical optimization iteratively finding the best fitting quadratic, and then exactly minimizing that quadratic.\n",
    "\n",
    "## A Little Matrix Calculus\n",
    "Derivatives of functions involving matrices turn out to be particularly nice.  This section can become notationally heavy, so may be skipped in a first reading, but it is useful to know how derivatives of functions involving common matrix operations are often much cleaner than one might initially anticipate, particularly given how central matrix operations are to deep learning applications.\n",
    "\n",
    "Let's begin with an example.  Suppose that we have some fixed column vector $\\boldsymbol{\\beta}$, and we want to take the product function $f(\\mathbf{x}) = \\boldsymbol{\\beta}^\\top\\mathbf{x}$, and understand how the dot product changes when we change $\\mathbf{x}$.\n",
    "\n",
    "A bit of notation that will be useful when working with matrix derivatives in ML is called the *denominator layout matrix derivative* where we assemble our partial derivatives into the shape of whatever vector, matrix, or tensor is in the denominator of the differential.  In this case, we will write\n",
    "\n",
    "$$\n",
    "\\frac{df}{d\\mathbf{x}} = \\begin{bmatrix}\n",
    "\\frac{df}{dx_1} \\\\\n",
    "\\vdots \\\\\n",
    "\\frac{df}{dx_n}\n",
    "\\end{bmatrix},\n",
    "$$\n",
    "\n",
    "where we matched the shape of the column vector $\\mathbf{x}$.\n",
    "\n",
    "If we write out our function into components this is\n",
    "\n",
    "$$\n",
    "f(\\mathbf{x}) = \\sum_{i = 1}^{n} \\beta_ix_i = \\beta_1x_1 + \\cdots + \\beta_nx_n.\n",
    "$$\n",
    "\n",
    "If we now take the partial derivative with respect to say $\\beta_1$, note that everything is zero but the first term, which is just $x_1$ multiplied by $\\beta_1$, so we obtain that\n",
    "\n",
    "$$\n",
    "\\frac{df}{dx_1} = \\beta_1,\n",
    "$$\n",
    "\n",
    "or more generally that\n",
    "\n",
    "$$\n",
    "\\frac{df}{dx_i} = \\beta_i.\n",
    "$$\n",
    "\n",
    "We can now reassemble this into a matrix to see\n",
    "\n",
    "$$\n",
    "\\frac{df}{d\\mathbf{x}} = \\begin{bmatrix}\n",
    "\\frac{df}{dx_1} \\\\\n",
    "\\vdots \\\\\n",
    "\\frac{df}{dx_n}\n",
    "\\end{bmatrix} = \\begin{bmatrix}\n",
    "\\beta_1 \\\\\n",
    "\\vdots \\\\\n",
    "\\beta_n\n",
    "\\end{bmatrix} = \\boldsymbol{\\beta}.\n",
    "$$\n",
    "\n",
    "This illustrates a few factors about matrix calculus that we will often counter throughout this section:\n",
    "\n",
    "* First, The computations will get rather involved.\n",
    "* Second, The final results are much cleaner than the intermediate process, and will always look similar to the single variable case.  In this case, note that $\\frac{d}{dx}(bx) = b$ and $\\frac{d}{d\\mathbf{x}} (\\boldsymbol{\\beta}^\\top\\mathbf{x}) = \\boldsymbol{\\beta}$ are both similar.\n",
    "* Third, transposes can often appear seemingly from nowhere.  The core reason for this is the convention that we match the shape of the denominator, thus when we multiply matrices, we will need to take transposes to match back to the shape of the original term.\n",
    "\n",
    "To keep building intuition, let's try a computation that is a little harder.  Suppose that we have a column vector $\\mathbf{x}$, and a square matrix $A$ and we want to compute\n",
    "\n",
    "$$\\frac{d}{d\\mathbf{x}}(\\mathbf{x}^\\top A \\mathbf{x}).$$\n",
    ":eqlabel:`eq_mat_goal_1`\n",
    "\n",
    "To drive towards easier to manipulate notation, let's consider this problem using Einstein notation.  In this case we can write the function as\n",
    "\n",
    "$$\n",
    "\\mathbf{x}^\\top A \\mathbf{x} = x_ia_{ij}x_j.\n",
    "$$\n",
    "\n",
    "To compute our derivative, we need to understand for every $k$, what is the value of\n",
    "\n",
    "$$\n",
    "\\frac{d}{dx_k}(\\mathbf{x}^\\top A \\mathbf{x}) = \\frac{d}{dx_k}x_ia_{ij}x_j.\n",
    "$$\n",
    "\n",
    "By the product rule, this is\n",
    "\n",
    "$$\n",
    "\\frac{d}{dx_k}x_ia_{ij}x_j = \\frac{dx_i}{dx_k}a_{ij}x_j + x_ia_{ij}\\frac{dx_j}{dx_k}.\n",
    "$$\n",
    "\n",
    "For a term like $\\frac{dx_i}{dx_k}$, it is not hard to see that this is one when $i=k$ and zero otherwise.  This means that every term where $i$ and $k$ are different vanish from this sum, so the only terms that remain in that first sum are the ones where $i=k$.  The same reasoning holds for the second term where we need $j=k$.  This gives\n",
    "\n",
    "$$\n",
    "\\frac{d}{dx_k}x_ia_{ij}x_j = a_{kj}x_j + x_ia_{ik}.\n",
    "$$\n",
    "\n",
    "Now, the names of the indices in Einstein notation are arbitrary---the fact that $i$ and $j$ are different is immaterial to this computation at this point, so we can re-index so that they both use $i$ to see that\n",
    "\n",
    "$$\n",
    "\\frac{d}{dx_k}x_ia_{ij}x_j = a_{ki}x_i + x_ia_{ik} = (a_{ki} + a_{ik})x_i.\n",
    "$$\n",
    "\n",
    "Now, here is where we start to need some practice to go further.  Let's try and identify this outcome in terms of matrix operations.  $a_{ki} + a_{ik}$ is the $k, i$-th component of $\\mathbf{A} + \\mathbf{A}^\\top$.  This gives\n",
    "\n",
    "$$\n",
    "\\frac{d}{dx_k}x_ia_{ij}x_j = [\\mathbf{A} + \\mathbf{A}^\\top]_{ki}x_i.\n",
    "$$\n",
    "\n",
    "Similarly, this term is now the product of the matrix $\\mathbf{A} + \\mathbf{A}^\\top$ by the vector $\\mathbf{x}$, so we see that\n",
    "\n",
    "$$\n",
    "\\left[\\frac{d}{d\\mathbf{x}}(\\mathbf{x}^\\top A \\mathbf{x})\\right]_k = \\frac{d}{dx_k}x_ia_{ij}x_j = [(\\mathbf{A} + \\mathbf{A}^\\top)\\mathbf{x}]_k.\n",
    "$$\n",
    "\n",
    "Thus, we see that the $k$-th entry of the desired derivative from :eqref:`eq_mat_goal_1` is just the $k$-th entry of the vector on the right, and thus the two are the same.  Thus yields\n",
    "\n",
    "$$\n",
    "\\frac{d}{d\\mathbf{x}}(\\mathbf{x}^\\top A \\mathbf{x}) = (\\mathbf{A} + \\mathbf{A}^\\top)\\mathbf{x}.\n",
    "$$\n",
    "\n",
    "This required significantly more work than our last one, but the final result is small.  More than that, consider the following computation for traditional single variable derivatives:\n",
    "\n",
    "$$\n",
    "\\frac{d}{dx}(xax) = \\frac{dx}{dx}ax + xa\\frac{dx}{dx} = (a+a)x.\n",
    "$$\n",
    "\n",
    "Equivalently $\\frac{d}{dx}(ax^2) = 2ax = (a+a)x$.  Again, we get a result that looks rather like the single variable result but with a transpose tossed in.\n",
    "\n",
    "At this point, the pattern should be looking rather suspicious, so let's try to figure out why.  When we take matrix derivatives like this, let's first assume that the expression we get will be another matrix expression: an expression we can write it in terms of products and sums of matrices and their transposes.  If such an expression exists, it will need to be true for all matrices.  In particular, it will need to be true of $1 \\times 1$ matrices, in which case the matrix product is just the product of the numbers, the matrix sum is just the sum, and the transpose does nothing at all!  In other words, whatever expression we get *must* match the single variable expression.  This means that, with some practice, one can often guess matrix derivatives just by knowing what the associated single variable expression must look like!\n",
    "\n",
    "Let's try this out.  Suppose that $\\mathbf{X}$ is a $n \\times m$ matrix, $\\mathbf{U}$ is an $n \\times r$ and $\\mathbf{V}$ is an $r \\times m$.  Let's try to compute\n",
    "\n",
    "$$\\frac{d}{d\\mathbf{V}} \\|\\mathbf{X} - \\mathbf{U}\\mathbf{V}\\|_2^{2} = \\;?$$\n",
    ":eqlabel:`eq_mat_goal_2`\n",
    "\n",
    "This computation is important in an area called matrix factorization.  For us, however, it is just a derivative to compute.  Let's try to imagine what this would be for $1\\times1$ matrices.  In that case, we get the expression\n",
    "\n",
    "$$\n",
    "\\frac{d}{dv} (x-uv)^{2}= -2(x-uv)u,\n",
    "$$\n",
    "\n",
    "where, the derivative is rather standard.  If we try to convert this back into a matrix expression we get\n",
    "\n",
    "$$\n",
    "\\frac{d}{d\\mathbf{V}} \\|\\mathbf{X} - \\mathbf{U}\\mathbf{V}\\|_2^{2}= -2(\\mathbf{X} - \\mathbf{U}\\mathbf{V})\\mathbf{U}.\n",
    "$$\n",
    "\n",
    "However, if we look at this it does not quite work.  Recall that $\\mathbf{X}$ is $n \\times m$, as is $\\mathbf{U}\\mathbf{V}$, so the matrix $2(\\mathbf{X} - \\mathbf{U}\\mathbf{V})$ is $n \\times m$.  On the other hand $\\mathbf{U}$ is $n \\times r$, and we cannot multiply a $n \\times m$ and a $n \\times r$ matrix since the dimensions do not match!\n",
    "\n",
    "We want to get $\\frac{d}{d\\mathbf{V}}$, which is the same shape as $\\mathbf{V}$, which is $r \\times m$.  So somehow we need to take a $n \\times m$ matrix and a $n \\times r$ matrix, multiply them together (perhaps with some transposes) to get a $r \\times m$. We can do this by multiplying $U^\\top$ by $(\\mathbf{X} - \\mathbf{U}\\mathbf{V})$.  Thus, we can guess the solution to :eqref:`eq_mat_goal_2` is\n",
    "\n",
    "$$\n",
    "\\frac{d}{d\\mathbf{V}} \\|\\mathbf{X} - \\mathbf{U}\\mathbf{V}\\|_2^{2}= -2\\mathbf{U}^\\top(\\mathbf{X} - \\mathbf{U}\\mathbf{V}).\n",
    "$$\n",
    "\n",
    "To show that this works, we would be remiss to not provide a detailed computation.  If we already believe that this rule-of-thumb works, feel free to skip past this derivation.  To compute\n",
    "\n",
    "$$\n",
    "\\frac{d}{d\\mathbf{V}} \\|\\mathbf{X} - \\mathbf{U}\\mathbf{V}\\|_2^2,\n",
    "$$\n",
    "\n",
    "we must find for every $a$, and $b$\n",
    "\n",
    "$$\n",
    "\\frac{d}{dv_{ab}} \\|\\mathbf{X} - \\mathbf{U}\\mathbf{V}\\|_2^{2}= \\frac{d}{dv_{ab}} \\sum_{i, j}\\left(x_{ij} - \\sum_k u_{ik}v_{kj}\\right)^2.\n",
    "$$\n",
    "\n",
    "Recalling that all entries of $\\mathbf{X}$ and $\\mathbf{U}$ are constants as far as $\\frac{d}{dv_{ab}}$ is concerned, we may push the derivative inside the sum, and apply the chain rule to the square to get\n",
    "\n",
    "$$\n",
    "\\frac{d}{dv_{ab}} \\|\\mathbf{X} - \\mathbf{U}\\mathbf{V}\\|_2^{2}= \\sum_{i, j}2\\left(x_{ij} - \\sum_k u_{ik}v_{kj}\\right)\\left(-\\sum_k u_{ik}\\frac{dv_{kj}}{dv_{ab}} \\right).\n",
    "$$\n",
    "\n",
    "As in the previous derivation, we may note that $\\frac{dv_{kj}}{dv_{ab}}$ is only non-zero if the $k=a$ and $j=b$.  If either of those conditions do not hold, the term in the sum is zero, and we may freely discard it.  We see that\n",
    "\n",
    "$$\n",
    "\\frac{d}{dv_{ab}} \\|\\mathbf{X} - \\mathbf{U}\\mathbf{V}\\|_2^{2}= -2\\sum_{i}\\left(x_{ib} - \\sum_k u_{ik}v_{kb}\\right)u_{ia}.\n",
    "$$\n",
    "\n",
    "An important subtlety here is that the requirement that $k=a$ does not occur inside the inner sum since that $k$ is a dummy variable which we are summing over inside the inner term.  For a notationally cleaner example, consider why\n",
    "\n",
    "$$\n",
    "\\frac{d}{dx_1} \\left(\\sum_i x_i \\right)^{2}= 2\\left(\\sum_i x_i \\right).\n",
    "$$\n",
    "\n",
    "From this point, we may start identifying components of the sum.  First,\n",
    "\n",
    "$$\n",
    "\\sum_k u_{ik}v_{kb} = [\\mathbf{U}\\mathbf{V}]_{ib}.\n",
    "$$\n",
    "\n",
    "So the entire expression in the inside of the sum is\n",
    "\n",
    "$$\n",
    "x_{ib} - \\sum_k u_{ik}v_{kb} = [\\mathbf{X}-\\mathbf{U}\\mathbf{V}]_{ib}.\n",
    "$$\n",
    "\n",
    "This means we may now write our derivative as\n",
    "\n",
    "$$\n",
    "\\frac{d}{dv_{ab}} \\|\\mathbf{X} - \\mathbf{U}\\mathbf{V}\\|_2^{2}= -2\\sum_{i}[\\mathbf{X}-\\mathbf{U}\\mathbf{V}]_{ib}u_{ia}.\n",
    "$$\n",
    "\n",
    "We want this to look like the $a, b$ element of a matrix so we can use the technique as in the previous example to arrive at a matrix expression, which means that we need to exchange the order of the indices on $u_{ia}$.  If we notice that $u_{ia} = [\\mathbf{U}^\\top]_{ai}$, we can then write\n",
    "\n",
    "$$\n",
    "\\frac{d}{dv_{ab}} \\|\\mathbf{X} - \\mathbf{U}\\mathbf{V}\\|_2^{2}= -2\\sum_{i} [\\mathbf{U}^\\top]_{ai}[\\mathbf{X}-\\mathbf{U}\\mathbf{V}]_{ib}.\n",
    "$$\n",
    "\n",
    "This is a matrix product, and thus we can conclude that\n",
    "\n",
    "$$\n",
    "\\frac{d}{dv_{ab}} \\|\\mathbf{X} - \\mathbf{U}\\mathbf{V}\\|_2^{2}= -2[\\mathbf{U}^\\top(\\mathbf{X}-\\mathbf{U}\\mathbf{V})]_{ab}.\n",
    "$$\n",
    "\n",
    "and thus we may write the solution to :eqref:`eq_mat_goal_2`\n",
    "\n",
    "$$\n",
    "\\frac{d}{d\\mathbf{V}} \\|\\mathbf{X} - \\mathbf{U}\\mathbf{V}\\|_2^{2}= -2\\mathbf{U}^\\top(\\mathbf{X} - \\mathbf{U}\\mathbf{V}).\n",
    "$$\n",
    "\n",
    "This matches the solution we guessed above!\n",
    "\n",
    "It is reasonable to ask at this point, \"Why can I not just write down matrix versions of all the calculus rules I have learned?  It is clear this is still mechanical.  Why do we not just get it over with!\"  And indeed there are such rules and :cite:`Petersen.Pedersen.ea.2008` provides an excellent summary.  However, due to the plethora of ways matrix operations can be combined compared to single values, there are many more matrix derivative rules than single variable ones.  It is often the case that it is best to work with the indices, or leave it up to automatic differentiation when appropriate.\n",
    "\n",
    "## Summary\n",
    "\n",
    "* In higher dimensions, we can define gradients which serve the same purpose as derivatives in one dimension.  These allow us to see how a multi-variable function changes when we make an arbitrary small change to the inputs.\n",
    "* The backpropagation algorithm can be seen to be a method of organizing the multi-variable chain rule to allow for the efficient computation of many partial derivatives.\n",
    "* Matrix calculus allows us to write the derivatives of matrix expressions in concise ways.\n",
    "\n",
    "## Exercises\n",
    "1. Given a column vector $\\boldsymbol{\\beta}$, compute the derivatives of both $f(\\mathbf{x}) = \\boldsymbol{\\beta}^\\top\\mathbf{x}$ and $g(\\mathbf{x}) = \\mathbf{x}^\\top\\boldsymbol{\\beta}$.  Why do you get the same answer?\n",
    "2. Let $\\mathbf{v}$ be an $n$ dimension vector. What is $\\frac{\\partial}{\\partial\\mathbf{v}}\\|\\mathbf{v}\\|_2$?\n",
    "3. Let $L(x, y) = \\log(e^x + e^y)$.  Compute the gradient.  What is the sum of the components of the gradient?\n",
    "4. Let $f(x, y) = x^2y + xy^2$. Show that the only critical point is $(0,0)$. By considering $f(x, x)$, determine if $(0,0)$ is a maximum, minimum, or neither.\n",
    "5. Suppose that we are minimizing a function $f(\\mathbf{x}) = g(\\mathbf{x}) + h(\\mathbf{x})$.  How can we geometrically interpret the condition of $\\nabla f = 0$ in terms of $g$ and $h$?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "720d5539",
   "metadata": {
    "origin_pos": 22,
    "tab": [
     "pytorch"
    ]
   },
   "source": [
    "[Discussions](https://discuss.d2l.ai/t/1090)\n"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  },
  "required_libs": []
 },
 "nbformat": 4,
 "nbformat_minor": 5
}